* [RFC PATCH v2 10/37] KVM: guest_memfd: Handle lru_add fbatch refcounts during conversion safety check
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao, Ackerley Tng
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
When checking if a guest_memfd folio is safe for conversion, its refcount
is examined. A folio may be present in a per-CPU lru_add fbatch, which
temporarily increases its refcount. This can lead to a false positive,
incorrectly indicating that the folio is in use and preventing the
conversion, even if it is otherwise safe. The conversion process might not
be on the same CPU that holds the folio in its fbatch, making a simple
per-CPU check insufficient.
To address this, drain all CPUs' lru_add fbatches if an unexpectedly high
refcount is encountered during the safety check. This is performed at most
once per conversion request.
guest_memfd folios are unevictable, so they can only reside in the lru_add
fbatch. If the folio's refcount is still unsafe after draining, then the
conversion is truly deemed unsafe.
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
virt/kvm/guest_memfd.c | 22 ++++++++++++++++------
1 file changed, 16 insertions(+), 6 deletions(-)
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 1cd8024cdb39..a9d12abfacb5 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -8,6 +8,7 @@
#include <linux/mempolicy.h>
#include <linux/pseudo_fs.h>
#include <linux/pagemap.h>
+#include <linux/swap.h>
#include "kvm_mm.h"
@@ -571,25 +572,34 @@ unsigned long kvm_gmem_get_memory_attributes(struct kvm *kvm, gfn_t gfn)
}
EXPORT_SYMBOL_GPL(kvm_gmem_get_memory_attributes);
-static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start,
- size_t nr_pages, pgoff_t *err_index)
+static bool kvm_gmem_is_safe_for_conversion(struct inode *inode,
+ pgoff_t start, size_t nr_pages,
+ pgoff_t *err_index)
{
struct address_space *mapping = inode->i_mapping;
const int filemap_get_folios_refcount = 1;
pgoff_t last = start + nr_pages - 1;
struct folio_batch fbatch;
+ bool lru_drained = false;
bool safe = true;
int i;
folio_batch_init(&fbatch);
while (safe && filemap_get_folios(mapping, &start, last, &fbatch)) {
- for (i = 0; i < folio_batch_count(&fbatch); ++i) {
+ for (i = 0; i < folio_batch_count(&fbatch);) {
struct folio *folio = fbatch.folios[i];
- if (folio_ref_count(folio) !=
- folio_nr_pages(folio) + filemap_get_folios_refcount) {
- safe = false;
+ safe = (folio_ref_count(folio) ==
+ folio_nr_pages(folio) +
+ filemap_get_folios_refcount);
+
+ if (safe) {
+ ++i;
+ } else if (!lru_drained) {
+ lru_add_drain_all();
+ lru_drained = true;
+ } else {
*err_index = folio->index;
break;
}
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 09/37] KVM: guest_memfd: Add support for KVM_SET_MEMORY_ATTRIBUTES2
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao, Ackerley Tng
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
For shared to private conversions, if refcounts on any of the folios
within the range are elevated, fail the conversion with -EAGAIN.
At the point of shared to private conversion, all folios in range are
also unmapped. The filemap_invalidate_lock() is held, so no faulting
can occur. Hence, from that point on, only transient refcounts can be
taken on the folios associated with that guest_memfd.
Hence, it is safe to do the conversion from shared to private.
After conversion is complete, refcounts may become elevated, but that
is fine since users of transient refcounts don't actually access
memory.
For private to shared conversions, there are no refcount checks, since
the guest is the only user of private pages, and guest_memfd will be the
only holder of refcounts on private pages.
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
Documentation/virt/kvm/api.rst | 48 +++++++-
include/linux/kvm_host.h | 10 ++
include/uapi/linux/kvm.h | 9 +-
virt/kvm/Kconfig | 2 +-
virt/kvm/guest_memfd.c | 210 +++++++++++++++++++++++++++++++--
virt/kvm/kvm_main.c | 15 +--
6 files changed, 263 insertions(+), 31 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 23ec0b0c3e22..26e80745c8b4 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -117,7 +117,7 @@ description:
x86 includes both i386 and x86_64.
Type:
- system, vm, or vcpu.
+ system, vm, vcpu or guest_memfd.
Parameters:
what parameters are accepted by the ioctl.
@@ -6523,11 +6523,22 @@ the capability to be present.
---------------------------------
:Capability: KVM_CAP_MEMORY_ATTRIBUTES2
-:Architectures: x86
-:Type: vm ioctl
+:Architectures: all
+:Type: vm, guest_memfd ioctl
:Parameters: struct kvm_memory_attributes2 (in/out)
:Returns: 0 on success, <0 on error
+Errors:
+
+ ========== ===============================================================
+ EINVAL The specified `offset` or `size` were invalid (e.g. not
+ page aligned, causes an overflow, or size is zero).
+ EFAULT The parameter address was invalid.
+ EAGAIN Some page within requested range had unexpected refcounts. The
+ offset of the page will be returned in `error_offset`.
+ ENOMEM Ran out of memory trying to track private/shared state
+ ========== ===============================================================
+
KVM_SET_MEMORY_ATTRIBUTES2 is an extension to
KVM_SET_MEMORY_ATTRIBUTES that supports returning (writing) values to
userspace. The original (pre-extension) fields are shared with
@@ -6538,15 +6549,42 @@ Attribute values are shared with KVM_SET_MEMORY_ATTRIBUTES.
::
struct kvm_memory_attributes2 {
- __u64 address;
+ /* in */
+ union {
+ __u64 address;
+ __u64 offset;
+ };
__u64 size;
__u64 attributes;
__u64 flags;
- __u64 reserved[12];
+ /* out */
+ __u64 error_offset;
+ __u64 reserved[11];
};
#define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3)
+Set attributes for a range of offsets within a guest_memfd to
+KVM_MEMORY_ATTRIBUTE_PRIVATE to limit the specified guest_memfd backed
+memory range for guest_use. Even if KVM_CAP_GUEST_MEMFD_MMAP is
+supported, after a successful call to set
+KVM_MEMORY_ATTRIBUTE_PRIVATE, the requested range will not be mappable
+into host userspace and will only be mappable by the guest.
+
+To allow the range to be mappable into host userspace again, call
+KVM_SET_MEMORY_ATTRIBUTES2 on the guest_memfd again with
+KVM_MEMORY_ATTRIBUTE_PRIVATE unset.
+
+If this ioctl returns -EAGAIN, the offset of the page with unexpected
+refcounts will be returned in `error_offset`. This can occur if there
+are transient refcounts on the pages, taken by other parts of the
+kernel.
+
+Userspace is expected to figure out how to remove all known refcounts
+on the shared pages, such as refcounts taken by get_user_pages(), and
+try the ioctl again. A possible source of these long term refcounts is
+if the guest_memfd memory was pinned in IOMMU page tables.
+
See also: :ref: `KVM_SET_MEMORY_ATTRIBUTES`.
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 8f1e10a503f4..8276187f5e4c 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -2516,6 +2516,16 @@ static inline bool kvm_memslot_is_gmem_only(const struct kvm_memory_slot *slot)
}
#ifdef CONFIG_KVM_MEMORY_ATTRIBUTES
+static inline u64 kvm_supported_mem_attributes(struct kvm *kvm)
+{
+#ifdef kvm_arch_has_private_mem
+ if (!kvm || kvm_arch_has_private_mem(kvm))
+ return KVM_MEMORY_ATTRIBUTE_PRIVATE;
+#endif
+
+ return 0;
+}
+
typedef unsigned long (kvm_get_memory_attributes_t)(struct kvm *kvm, gfn_t gfn);
DECLARE_STATIC_CALL(__kvm_get_memory_attributes, kvm_get_memory_attributes_t);
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 7f8dac4f4fd3..fba5e0c851f2 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -975,6 +975,7 @@ struct kvm_enable_cap {
#define KVM_CAP_ARM_SEA_TO_USER 245
#define KVM_CAP_S390_USER_OPEREXEC 246
#define KVM_CAP_MEMORY_ATTRIBUTES2 247
+#define KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES 248
struct kvm_irq_routing_irqchip {
__u32 irqchip;
@@ -1612,11 +1613,15 @@ struct kvm_memory_attributes {
#define KVM_SET_MEMORY_ATTRIBUTES2 _IOWR(KVMIO, 0xd2, struct kvm_memory_attributes2)
struct kvm_memory_attributes2 {
- __u64 address;
+ union {
+ __u64 address;
+ __u64 offset;
+ };
__u64 size;
__u64 attributes;
__u64 flags;
- __u64 reserved[12];
+ __u64 error_offset;
+ __u64 reserved[11];
};
#define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3)
diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
index 02c4c653bb31..f42bc6e7de44 100644
--- a/virt/kvm/Kconfig
+++ b/virt/kvm/Kconfig
@@ -114,7 +114,7 @@ config KVM_VM_MEMORY_ATTRIBUTES
bool
config KVM_GUEST_MEMFD
- depends on KVM_GENERIC_MMU_NOTIFIER
+ select KVM_MEMORY_ATTRIBUTES
select XARRAY_MULTI
bool
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 6b3477b226ad..1cd8024cdb39 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -86,6 +86,23 @@ static bool kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index)
return !kvm_gmem_is_private_mem(inode, index);
}
+static bool kvm_gmem_range_has_attributes(struct maple_tree *mt,
+ pgoff_t index, size_t nr_pages,
+ u64 attributes)
+{
+ pgoff_t end = index + nr_pages - 1;
+ void *entry;
+
+ lockdep_assert(mt_lock_is_held(mt));
+
+ mt_for_each(mt, entry, index, end) {
+ if (xa_to_value(entry) != attributes)
+ return false;
+ }
+
+ return true;
+}
+
static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,
pgoff_t index, struct folio *folio)
{
@@ -183,10 +200,12 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
static enum kvm_gfn_range_filter kvm_gmem_get_invalidate_filter(struct inode *inode)
{
- if (GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED)
- return KVM_FILTER_SHARED;
-
- return KVM_FILTER_PRIVATE;
+ /*
+ * TODO: Limit invalidations based on the to-be-invalidated range, i.e.
+ * invalidate shared/private if and only if there can possibly be
+ * such mappings.
+ */
+ return KVM_FILTER_SHARED | KVM_FILTER_PRIVATE;
}
static void __kvm_gmem_invalidate_begin(struct gmem_file *f, pgoff_t start,
@@ -552,11 +571,183 @@ unsigned long kvm_gmem_get_memory_attributes(struct kvm *kvm, gfn_t gfn)
}
EXPORT_SYMBOL_GPL(kvm_gmem_get_memory_attributes);
+static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start,
+ size_t nr_pages, pgoff_t *err_index)
+{
+ struct address_space *mapping = inode->i_mapping;
+ const int filemap_get_folios_refcount = 1;
+ pgoff_t last = start + nr_pages - 1;
+ struct folio_batch fbatch;
+ bool safe = true;
+ int i;
+
+ folio_batch_init(&fbatch);
+ while (safe && filemap_get_folios(mapping, &start, last, &fbatch)) {
+
+ for (i = 0; i < folio_batch_count(&fbatch); ++i) {
+ struct folio *folio = fbatch.folios[i];
+
+ if (folio_ref_count(folio) !=
+ folio_nr_pages(folio) + filemap_get_folios_refcount) {
+ safe = false;
+ *err_index = folio->index;
+ break;
+ }
+ }
+
+ folio_batch_release(&fbatch);
+ }
+
+ return safe;
+}
+
+/*
+ * Preallocate memory for attributes to be stored on a maple tree, pointed to
+ * by mas. Adjacent ranges with attributes identical to the new attributes
+ * will be merged. Also sets mas's bounds up for storing attributes.
+ *
+ * This maintains the invariant that ranges with the same attributes will
+ * always be merged.
+ */
+static int kvm_gmem_mas_preallocate(struct ma_state *mas, u64 attributes,
+ pgoff_t start, size_t nr_pages)
+{
+ pgoff_t end = start + nr_pages;
+ pgoff_t last = end - 1;
+ void *entry;
+
+ /* Try extending range. entry is NULL on overflow/wrap-around. */
+ mas_set_range(mas, end, end);
+ entry = mas_find(mas, end);
+ if (entry && xa_to_value(entry) == attributes)
+ last = mas->last;
+
+ if (start > 0) {
+ mas_set_range(mas, start - 1, start - 1);
+ entry = mas_find(mas, start - 1);
+ if (entry && xa_to_value(entry) == attributes)
+ start = mas->index;
+ }
+
+ mas_set_range(mas, start, last);
+ return mas_preallocate(mas, xa_mk_value(attributes), GFP_KERNEL);
+}
+
+static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
+ size_t nr_pages, uint64_t attrs,
+ pgoff_t *err_index)
+{
+ struct address_space *mapping = inode->i_mapping;
+ struct gmem_inode *gi = GMEM_I(inode);
+ pgoff_t end = start + nr_pages;
+ struct maple_tree *mt;
+ struct ma_state mas;
+ int r;
+
+ mt = &gi->attributes;
+
+ filemap_invalidate_lock(mapping);
+
+ mas_init(&mas, mt, start);
+
+ if (kvm_gmem_range_has_attributes(mt, start, nr_pages, attrs)) {
+ r = 0;
+ goto out;
+ }
+
+ r = kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages);
+ if (r) {
+ *err_index = start;
+ goto out;
+ }
+
+ if (attrs & KVM_MEMORY_ATTRIBUTE_PRIVATE) {
+ unmap_mapping_pages(mapping, start, nr_pages, false);
+
+ if (!kvm_gmem_is_safe_for_conversion(inode, start, nr_pages,
+ err_index)) {
+ mas_destroy(&mas);
+ r = -EAGAIN;
+ goto out;
+ }
+ }
+
+ kvm_gmem_invalidate_begin(inode, start, end);
+
+ mas_store_prealloc(&mas, xa_mk_value(attrs));
+
+ kvm_gmem_invalidate_end(inode, start, end);
+out:
+ filemap_invalidate_unlock(mapping);
+ return r;
+}
+
+static long kvm_gmem_set_attributes(struct file *file, void __user *argp)
+{
+ struct gmem_file *f = file->private_data;
+ struct inode *inode = file_inode(file);
+ struct kvm_memory_attributes2 attrs;
+ pgoff_t err_index;
+ size_t nr_pages;
+ pgoff_t index;
+ int i, r;
+
+ if (copy_from_user(&attrs, argp, sizeof(attrs)))
+ return -EFAULT;
+
+ if (attrs.flags)
+ return -EINVAL;
+ if (attrs.error_offset)
+ return -EINVAL;
+ for (i = 0; i < ARRAY_SIZE(attrs.reserved); i++) {
+ if (attrs.reserved[i])
+ return -EINVAL;
+ }
+ if (attrs.attributes & ~kvm_supported_mem_attributes(f->kvm))
+ return -EINVAL;
+ if (attrs.size == 0 || attrs.offset + attrs.size < attrs.offset)
+ return -EINVAL;
+ if (!PAGE_ALIGNED(attrs.offset) || !PAGE_ALIGNED(attrs.size))
+ return -EINVAL;
+
+ if (attrs.offset >= inode->i_size ||
+ attrs.offset + attrs.size > inode->i_size)
+ return -EINVAL;
+
+ nr_pages = attrs.size >> PAGE_SHIFT;
+ index = attrs.offset >> PAGE_SHIFT;
+ r = __kvm_gmem_set_attributes(inode, index, nr_pages, attrs.attributes,
+ &err_index);
+ if (r) {
+ attrs.error_offset = err_index << PAGE_SHIFT;
+
+ if (copy_to_user(argp, &attrs, sizeof(attrs)))
+ return -EFAULT;
+ }
+
+ return r;
+}
+
+static long kvm_gmem_ioctl(struct file *file, unsigned int ioctl,
+ unsigned long arg)
+{
+ switch (ioctl) {
+ case KVM_SET_MEMORY_ATTRIBUTES2:
+ if (vm_memory_attributes)
+ return -ENOTTY;
+
+ return kvm_gmem_set_attributes(file, (void __user *)arg);
+ default:
+ return -ENOTTY;
+ }
+}
+
static struct file_operations kvm_gmem_fops = {
.mmap = kvm_gmem_mmap,
.open = generic_file_open,
.release = kvm_gmem_release,
.fallocate = kvm_gmem_fallocate,
+ .unlocked_ioctl = kvm_gmem_ioctl,
};
static int kvm_gmem_migrate_folio(struct address_space *mapping,
@@ -937,20 +1128,13 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_get_pfn);
static bool kvm_gmem_range_is_private(struct gmem_inode *gi, pgoff_t index,
size_t nr_pages, struct kvm *kvm, gfn_t gfn)
{
- pgoff_t end = index + nr_pages - 1;
- void *entry;
-
if (vm_memory_attributes)
return kvm_range_has_vm_memory_attributes(kvm, gfn, gfn + nr_pages,
KVM_MEMORY_ATTRIBUTE_PRIVATE,
KVM_MEMORY_ATTRIBUTE_PRIVATE);
- mt_for_each(&gi->attributes, entry, index, end) {
- if (xa_to_value(entry) != attributes)
- return false;
- }
-
- return true;
+ return kvm_gmem_range_has_attributes(&gi->attributes, index, nr_pages,
+ KVM_MEMORY_ATTRIBUTE_PRIVATE);
}
long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long npages,
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index ad70101c2e3f..cf56cc892e7c 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2451,16 +2451,6 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm,
#endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */
#ifdef CONFIG_KVM_MEMORY_ATTRIBUTES
-static u64 kvm_supported_mem_attributes(struct kvm *kvm)
-{
-#ifdef kvm_arch_has_private_mem
- if (!kvm || kvm_arch_has_private_mem(kvm))
- return KVM_MEMORY_ATTRIBUTE_PRIVATE;
-#endif
-
- return 0;
-}
-
#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
static unsigned long kvm_get_vm_memory_attributes(struct kvm *kvm, gfn_t gfn)
{
@@ -4993,6 +4983,11 @@ static int kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)
return 1;
case KVM_CAP_GUEST_MEMFD_FLAGS:
return kvm_gmem_get_supported_flags(kvm);
+ case KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES:
+ if (vm_memory_attributes)
+ return 0;
+
+ return kvm_supported_mem_attributes(kvm);
#endif
default:
break;
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 08/37] KVM: guest_memfd: Enable INIT_SHARED on guest_memfd for x86 Coco VMs
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
From: Sean Christopherson <seanjc@google.com>
Now that guest_memfd supports tracking private vs. shared within gmem
itself, allow userspace to specify INIT_SHARED on a guest_memfd instance
for x86 Confidential Computing (CoCo) VMs, so long as per-VM attributes
are disabled, i.e. when it's actually possible for a guest_memfd instance
to contain shared memory.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b2e93f836dca..6518cdb4569f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -13984,14 +13984,13 @@ bool kvm_arch_no_poll(struct kvm_vcpu *vcpu)
}
#ifdef CONFIG_KVM_GUEST_MEMFD
-/*
- * KVM doesn't yet support initializing guest_memfd memory as shared for VMs
- * with private memory (the private vs. shared tracking needs to be moved into
- * guest_memfd).
- */
bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)
{
- return !kvm_arch_has_private_mem(kvm);
+ /*
+ * INIT_SHARED isn't supported if the memory attributes are per-VM,
+ * in which case guest_memfd can _only_ be used for private memory.
+ */
+ return !vm_memory_attributes || !kvm_arch_has_private_mem(kvm);
}
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 07/37] KVM: Introduce KVM_SET_MEMORY_ATTRIBUTES2
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao, Ackerley Tng
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
Introduce a "version 2" of KVM_SET_MEMORY_ATTRIBUTES to support returning
information back to userspace.
This new ioctl and structure will, in a later patch, be shared as a
guest_memfd ioctl, where the padding in the new kvm_memory_attributes2
structure will be for writing the response from the guest_memfd ioctl to
userspace.
A new ioctl is necessary for these reasons:
1. KVM_SET_MEMORY_ATTRIBUTES is currently a write-only ioctl and does not
allow userspace to read fields. There's nothing in code (yet?) that
validates this, but using _IOWR for consistency would be prudent.
2. KVM_SET_MEMORY_ATTRIBUTES, when used as a guest_memfd ioctl, will need
an additional field to provide userspace with more error details.
Alternatively, a completely new ioctl could be defined, unrelated to
KVM_SET_MEMORY_ATTRIBUTES, but using the same ioctl number and struct for
the vm and guest_memfd ioctls streamlines the interface for userspace. In
addition, any memory attributes, implemented on the vm or guest_memfd
ioctl, can be easily shared with the other.
Add KVM_CAP_MEMORY_ATTRIBUTES2 to indicate that struct
kvm_memory_attributes2 exists and can be used either with
KVM_SET_MEMORY_ATTRIBUTES2 via the vm or guest_memfd ioctl.
Since KVM_SET_MEMORY_ATTRIBUTES2 is not limited to be used only with the vm
ioctl, return 1 for KVM_CAP_MEMORY_ATTRIBUTES2 as long as struct
kvm_memory_attributes2 and KVM_SET_MEMORY_ATTRIBUTES2 can be
used. KVM_CAP_MEMORY_ATTRIBUTES must still be used to actually get valid
attributes.
Handle KVM_CAP_MEMORY_ATTRIBUTES2 and return 1 regardless of
CONFIG_KVM_VM_MEMORY_ATTRIBUTES, since KVM_SET_MEMORY_ATTRIBUTES2 is not
limited to a vm ioctl and can also be used with the guest_memfd ioctl.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
Documentation/virt/kvm/api.rst | 32 +++++++++++++++++++++++++++++++
include/uapi/linux/kvm.h | 12 ++++++++++++
virt/kvm/kvm_main.c | 35 +++++++++++++++++++++++++++++++---
3 files changed, 76 insertions(+), 3 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 01a3abef8abb..23ec0b0c3e22 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6360,6 +6360,8 @@ S390:
Returns -EINVAL if the VM has the KVM_VM_S390_UCONTROL flag set.
Returns -EINVAL if called on a protected VM.
+.. _KVM_SET_MEMORY_ATTRIBUTES:
+
4.141 KVM_SET_MEMORY_ATTRIBUTES
-------------------------------
@@ -6517,6 +6519,36 @@ the capability to be present.
`flags` must currently be zero.
+4.144 KVM_SET_MEMORY_ATTRIBUTES2
+---------------------------------
+
+:Capability: KVM_CAP_MEMORY_ATTRIBUTES2
+:Architectures: x86
+:Type: vm ioctl
+:Parameters: struct kvm_memory_attributes2 (in/out)
+:Returns: 0 on success, <0 on error
+
+KVM_SET_MEMORY_ATTRIBUTES2 is an extension to
+KVM_SET_MEMORY_ATTRIBUTES that supports returning (writing) values to
+userspace. The original (pre-extension) fields are shared with
+KVM_SET_MEMORY_ATTRIBUTES identically.
+
+Attribute values are shared with KVM_SET_MEMORY_ATTRIBUTES.
+
+::
+
+ struct kvm_memory_attributes2 {
+ __u64 address;
+ __u64 size;
+ __u64 attributes;
+ __u64 flags;
+ __u64 reserved[12];
+ };
+
+ #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3)
+
+See also: :ref: `KVM_SET_MEMORY_ATTRIBUTES`.
+
.. _kvm_run:
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index dddb781b0507..7f8dac4f4fd3 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -974,6 +974,7 @@ struct kvm_enable_cap {
#define KVM_CAP_GUEST_MEMFD_FLAGS 244
#define KVM_CAP_ARM_SEA_TO_USER 245
#define KVM_CAP_S390_USER_OPEREXEC 246
+#define KVM_CAP_MEMORY_ATTRIBUTES2 247
struct kvm_irq_routing_irqchip {
__u32 irqchip;
@@ -1607,6 +1608,17 @@ struct kvm_memory_attributes {
__u64 flags;
};
+/* Available with KVM_CAP_MEMORY_ATTRIBUTES2 */
+#define KVM_SET_MEMORY_ATTRIBUTES2 _IOWR(KVMIO, 0xd2, struct kvm_memory_attributes2)
+
+struct kvm_memory_attributes2 {
+ __u64 address;
+ __u64 size;
+ __u64 attributes;
+ __u64 flags;
+ __u64 reserved[12];
+};
+
#define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3)
#define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest_memfd)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 666d1f7fbf07..ad70101c2e3f 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2637,7 +2637,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
return r;
}
static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm,
- struct kvm_memory_attributes *attrs)
+ struct kvm_memory_attributes2 *attrs)
{
gfn_t start, end;
@@ -4981,6 +4981,7 @@ static int kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)
case KVM_CAP_DEVICE_CTRL:
return 1;
#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+ case KVM_CAP_MEMORY_ATTRIBUTES2:
case KVM_CAP_MEMORY_ATTRIBUTES:
if (!vm_memory_attributes)
return 0;
@@ -5206,6 +5207,14 @@ do { \
sizeof_field(struct kvm_userspace_memory_region2, field)); \
} while (0)
+#define SANITY_CHECK_MEMORY_ATTRIBUTES_FIELD(field) \
+do { \
+ BUILD_BUG_ON(offsetof(struct kvm_memory_attributes, field) != \
+ offsetof(struct kvm_memory_attributes2, field)); \
+ BUILD_BUG_ON(sizeof_field(struct kvm_memory_attributes, field) != \
+ sizeof_field(struct kvm_memory_attributes2, field)); \
+} while (0)
+
static long kvm_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -5388,15 +5397,35 @@ static long kvm_vm_ioctl(struct file *filp,
}
#endif /* CONFIG_HAVE_KVM_IRQ_ROUTING */
#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+ case KVM_SET_MEMORY_ATTRIBUTES2:
case KVM_SET_MEMORY_ATTRIBUTES: {
- struct kvm_memory_attributes attrs;
+ struct kvm_memory_attributes2 attrs;
+ unsigned long size;
+
+ if (ioctl == KVM_SET_MEMORY_ATTRIBUTES) {
+ /*
+ * Fields beyond struct kvm_memory_attributes shouldn't
+ * be accessed, but avoid leaking kernel memory in case
+ * of a bug.
+ */
+ memset(&attrs, 0, sizeof(attrs));
+ size = sizeof(struct kvm_memory_attributes);
+ } else {
+ size = sizeof(struct kvm_memory_attributes2);
+ }
+
+ /* Ensure the common parts of the two structs are identical. */
+ SANITY_CHECK_MEMORY_ATTRIBUTES_FIELD(address);
+ SANITY_CHECK_MEMORY_ATTRIBUTES_FIELD(size);
+ SANITY_CHECK_MEMORY_ATTRIBUTES_FIELD(attributes);
+ SANITY_CHECK_MEMORY_ATTRIBUTES_FIELD(flags);
r = -ENOTTY;
if (!vm_memory_attributes)
goto out;
r = -EFAULT;
- if (copy_from_user(&attrs, argp, sizeof(attrs)))
+ if (copy_from_user(&attrs, argp, size))
goto out;
r = kvm_vm_ioctl_set_mem_attributes(kvm, &attrs);
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 06/37] KVM: guest_memfd: Update kvm_gmem_populate() to use gmem attributes
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao, Ackerley Tng
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
Update the guest_memfd populate() flow to pull memory attributes from the
gmem instance instead of the VM when KVM is not configured to track
shared/private status in the VM.
Rename the per-VM API to make it clear that it retrieves per-VM
attributes, i.e. is not suitable for use outside of flows that are
specific to generic per-VM attributes.
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/mmu/mmu.c | 2 +-
include/linux/kvm_host.h | 14 +++++++++++++-
virt/kvm/guest_memfd.c | 26 +++++++++++++++++++++++---
virt/kvm/kvm_main.c | 8 +++-----
4 files changed, 40 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index c7de8ff84fd2..25ab6b8901e2 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -7979,7 +7979,7 @@ static bool hugepage_has_attrs(struct kvm *kvm, struct kvm_memory_slot *slot,
const unsigned long end = start + KVM_PAGES_PER_HPAGE(level);
if (level == PG_LEVEL_2M)
- return kvm_range_has_memory_attributes(kvm, start, end, ~0, attrs);
+ return kvm_range_has_vm_memory_attributes(kvm, start, end, ~0, attrs);
for (gfn = start; gfn < end; gfn += KVM_PAGES_PER_HPAGE(level - 1)) {
if (hugepage_test_mixed(slot, gfn, level - 1) ||
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index c12ee89392d8..8f1e10a503f4 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -2536,12 +2536,24 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn)
#endif
#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
-bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
+extern bool vm_memory_attributes;
+bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
unsigned long mask, unsigned long attrs);
bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm,
struct kvm_gfn_range *range);
bool kvm_arch_post_set_memory_attributes(struct kvm *kvm,
struct kvm_gfn_range *range);
+#else
+#define vm_memory_attributes false
+static inline bool kvm_range_has_vm_memory_attributes(struct kvm *kvm,
+ gfn_t start, gfn_t end,
+ unsigned long mask,
+ unsigned long attrs)
+{
+ WARN_ONCE(1, "Unexpected call to kvm_range_has_vm_memory_attributes()");
+
+ return false;
+}
#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
unsigned long kvm_gmem_get_memory_attributes(struct kvm *kvm, gfn_t gfn);
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 810fec394075..6b3477b226ad 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -934,10 +934,30 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_get_pfn);
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE
+static bool kvm_gmem_range_is_private(struct gmem_inode *gi, pgoff_t index,
+ size_t nr_pages, struct kvm *kvm, gfn_t gfn)
+{
+ pgoff_t end = index + nr_pages - 1;
+ void *entry;
+
+ if (vm_memory_attributes)
+ return kvm_range_has_vm_memory_attributes(kvm, gfn, gfn + nr_pages,
+ KVM_MEMORY_ATTRIBUTE_PRIVATE,
+ KVM_MEMORY_ATTRIBUTE_PRIVATE);
+
+ mt_for_each(&gi->attributes, entry, index, end) {
+ if (xa_to_value(entry) != attributes)
+ return false;
+ }
+
+ return true;
+}
+
long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long npages,
kvm_gmem_populate_cb post_populate, void *opaque)
{
struct kvm_memory_slot *slot;
+ struct gmem_inode *gi;
void __user *p;
int ret = 0, max_order;
@@ -956,6 +976,8 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long
if (!file)
return -EFAULT;
+ gi = GMEM_I(file_inode(file));
+
filemap_invalidate_lock(file->f_mapping);
npages = min_t(ulong, slot->npages - (start_gfn - slot->base_gfn), npages);
@@ -989,9 +1011,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long
(npages - i) < (1 << max_order));
ret = -EINVAL;
- while (!kvm_range_has_memory_attributes(kvm, gfn, gfn + (1 << max_order),
- KVM_MEMORY_ATTRIBUTE_PRIVATE,
- KVM_MEMORY_ATTRIBUTE_PRIVATE)) {
+ while (!kvm_gmem_range_is_private(gi, index, 1 << max_order, kvm, gfn)) {
if (!max_order)
goto put_folio_and_exit;
max_order--;
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 5d06e2c74ae7..666d1f7fbf07 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -104,9 +104,7 @@ module_param(allow_unsafe_mappings, bool, 0444);
#ifdef CONFIG_KVM_MEMORY_ATTRIBUTES
#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
-static bool vm_memory_attributes = true;
-#else
-#define vm_memory_attributes false
+bool vm_memory_attributes = true;
#endif
DEFINE_STATIC_CALL_RET0(__kvm_get_memory_attributes, kvm_get_memory_attributes_t);
EXPORT_SYMBOL_FOR_KVM_INTERNAL(STATIC_CALL_KEY(__kvm_get_memory_attributes));
@@ -2473,7 +2471,7 @@ static unsigned long kvm_get_vm_memory_attributes(struct kvm *kvm, gfn_t gfn)
* Returns true if _all_ gfns in the range [@start, @end) have attributes
* such that the bits in @mask match @attrs.
*/
-bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
+bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
unsigned long mask, unsigned long attrs)
{
XA_STATE(xas, &kvm->mem_attr_array, start);
@@ -2607,7 +2605,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
mutex_lock(&kvm->slots_lock);
/* Nothing to do if the entire range has the desired attributes. */
- if (kvm_range_has_memory_attributes(kvm, start, end, ~0, attributes))
+ if (kvm_range_has_vm_memory_attributes(kvm, start, end, ~0, attributes))
goto out_unlock;
/*
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 05/37] KVM: guest_memfd: Wire up kvm_get_memory_attributes() to per-gmem attributes
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao, Ackerley Tng
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
From: Sean Christopherson <seanjc@google.com>
Implement kvm_gmem_get_memory_attributes() for guest_memfd to allow the KVM
core and architecture code to query per-GFN memory attributes.
kvm_gmem_get_memory_attributes() finds the memory slot for a given GFN and
queries the guest_memfd file's to determine if the page is marked as
private.
If vm_memory_attributes is not enabled, there is no shared/private tracking
at the VM level. Install the guest_memfd implementation as long as
guest_memfd is enabled to give guest_memfd a chance to respond on
attributes.
guest_memfd should look up attributes regardless of whether this memslot is
gmem-only since attributes are now tracked by gmem regardless of whether
mmap() is enabled.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Co-developed-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
include/linux/kvm_host.h | 2 ++
virt/kvm/guest_memfd.c | 37 +++++++++++++++++++++++++++++++++++++
virt/kvm/kvm_main.c | 3 +++
3 files changed, 42 insertions(+)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 0e7920a0f957..c12ee89392d8 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -2544,6 +2544,8 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *kvm,
struct kvm_gfn_range *range);
#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
+unsigned long kvm_gmem_get_memory_attributes(struct kvm *kvm, gfn_t gfn);
+
#ifdef CONFIG_KVM_GUEST_MEMFD
int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
gfn_t gfn, kvm_pfn_t *pfn, struct page **page,
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 7a970e5fab67..810fec394075 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -515,6 +515,43 @@ static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
return 0;
}
+unsigned long kvm_gmem_get_memory_attributes(struct kvm *kvm, gfn_t gfn)
+{
+ struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn);
+ struct inode *inode;
+ unsigned long attrs;
+
+ /*
+ * If this gfn has no associated memslot, there's no chance of the gfn
+ * being backed by private memory, since guest_memfd must be used for
+ * private memory, and guest_memfd must be associated with some memslot.
+ */
+ if (!slot)
+ return 0;
+
+ CLASS(gmem_get_file, file)(slot);
+ if (!file)
+ return 0;
+
+ inode = file_inode(file);
+
+ /*
+ * Acquire the filemap lock to ensure the mtree lookup gets a
+ * stable result. The caller _must_ still protect consumption
+ * of private vs. shared by checking
+ * mmu_invalidate_retry_gfn() under mmu_lock to serialize
+ * against ongoing attribute updates. Acquiring the filemap
+ * lock only ensures a stable _lookup_, the result can become
+ * stale as soon as the lock is dropped.
+ */
+ filemap_invalidate_lock_shared(inode->i_mapping);
+ attrs = kvm_gmem_get_attributes(inode, kvm_gmem_get_index(slot, gfn));
+ filemap_invalidate_unlock_shared(inode->i_mapping);
+
+ return attrs;
+}
+EXPORT_SYMBOL_GPL(kvm_gmem_get_memory_attributes);
+
static struct file_operations kvm_gmem_fops = {
.mmap = kvm_gmem_mmap,
.open = generic_file_open,
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 4dc9fd941ecb..5d06e2c74ae7 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2676,6 +2676,9 @@ static void kvm_init_memory_attributes(void)
if (vm_memory_attributes)
static_call_update(__kvm_get_memory_attributes,
kvm_get_vm_memory_attributes);
+ else if (IS_ENABLED(CONFIG_KVM_GUEST_MEMFD))
+ static_call_update(__kvm_get_memory_attributes,
+ kvm_gmem_get_memory_attributes);
else
static_call_update(__kvm_get_memory_attributes,
(void *)__static_call_return0);
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 04/37] KVM: Stub in ability to disable per-VM memory attribute tracking
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
From: Sean Christopherson <seanjc@google.com>
Introduce the basic infrastructure to allow per-VM memory attribute
tracking to be disabled. This will be built-upon in a later patch, where a
module param can disable per-VM memory attribute tracking.
Split the Kconfig option into a base KVM_MEMORY_ATTRIBUTES and the
existing KVM_VM_MEMORY_ATTRIBUTES. The base option provides the core
plumbing, while the latter enables the full per-VM tracking via an xarray
and the associated ioctls.
kvm_get_memory_attributes() now performs a static call that either looks up
kvm->mem_attr_array with CONFIG_KVM_VM_MEMORY_ATTRIBUTES is enabled, or
just returns 0 otherwise. The static call can be patched depending on
whether per-VM tracking is enabled by the CONFIG.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 2 +-
include/linux/kvm_host.h | 23 ++++++++++-------
virt/kvm/Kconfig | 6 ++++-
virt/kvm/kvm_main.c | 44 ++++++++++++++++++++++++++++++++-
4 files changed, 63 insertions(+), 12 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index d427cf9187c3..f8f0de0c367e 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -2307,7 +2307,7 @@ void kvm_configure_mmu(bool enable_tdp, int tdp_forced_root_level,
int tdp_max_root_level, int tdp_huge_page_level);
-#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_MEMORY_ATTRIBUTES
#define kvm_arch_has_private_mem(kvm) ((kvm)->arch.has_private_mem)
#endif
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index af2fcfff7692..0e7920a0f957 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -2515,19 +2515,15 @@ static inline bool kvm_memslot_is_gmem_only(const struct kvm_memory_slot *slot)
return slot->flags & KVM_MEMSLOT_GMEM_ONLY;
}
-#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_MEMORY_ATTRIBUTES
+typedef unsigned long (kvm_get_memory_attributes_t)(struct kvm *kvm, gfn_t gfn);
+DECLARE_STATIC_CALL(__kvm_get_memory_attributes, kvm_get_memory_attributes_t);
+
static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn_t gfn)
{
- return xa_to_value(xa_load(&kvm->mem_attr_array, gfn));
+ return static_call(__kvm_get_memory_attributes)(kvm, gfn);
}
-bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
- unsigned long mask, unsigned long attrs);
-bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm,
- struct kvm_gfn_range *range);
-bool kvm_arch_post_set_memory_attributes(struct kvm *kvm,
- struct kvm_gfn_range *range);
-
static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn)
{
return kvm_get_memory_attributes(kvm, gfn) & KVM_MEMORY_ATTRIBUTE_PRIVATE;
@@ -2537,6 +2533,15 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn)
{
return false;
}
+#endif
+
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
+ unsigned long mask, unsigned long attrs);
+bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm,
+ struct kvm_gfn_range *range);
+bool kvm_arch_post_set_memory_attributes(struct kvm *kvm,
+ struct kvm_gfn_range *range);
#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
#ifdef CONFIG_KVM_GUEST_MEMFD
diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
index c12dc19f0a5e..02c4c653bb31 100644
--- a/virt/kvm/Kconfig
+++ b/virt/kvm/Kconfig
@@ -105,10 +105,14 @@ config KVM_MMU_LOCKLESS_AGING
depends on KVM_GENERIC_MMU_NOTIFIER
bool
-config KVM_VM_MEMORY_ATTRIBUTES
+config KVM_MEMORY_ATTRIBUTES
depends on KVM_GENERIC_MMU_NOTIFIER
bool
+config KVM_VM_MEMORY_ATTRIBUTES
+ select KVM_MEMORY_ATTRIBUTES
+ bool
+
config KVM_GUEST_MEMFD
depends on KVM_GENERIC_MMU_NOTIFIER
select XARRAY_MULTI
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 26e0d532ba03..4dc9fd941ecb 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -102,6 +102,17 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(halt_poll_ns_shrink);
static bool allow_unsafe_mappings;
module_param(allow_unsafe_mappings, bool, 0444);
+#ifdef CONFIG_KVM_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+static bool vm_memory_attributes = true;
+#else
+#define vm_memory_attributes false
+#endif
+DEFINE_STATIC_CALL_RET0(__kvm_get_memory_attributes, kvm_get_memory_attributes_t);
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(STATIC_CALL_KEY(__kvm_get_memory_attributes));
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(STATIC_CALL_TRAMP(__kvm_get_memory_attributes));
+#endif
+
/*
* Ordering of locks:
*
@@ -2441,7 +2452,7 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm,
}
#endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */
-#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_MEMORY_ATTRIBUTES
static u64 kvm_supported_mem_attributes(struct kvm *kvm)
{
#ifdef kvm_arch_has_private_mem
@@ -2452,6 +2463,12 @@ static u64 kvm_supported_mem_attributes(struct kvm *kvm)
return 0;
}
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+static unsigned long kvm_get_vm_memory_attributes(struct kvm *kvm, gfn_t gfn)
+{
+ return xa_to_value(xa_load(&kvm->mem_attr_array, gfn));
+}
+
/*
* Returns true if _all_ gfns in the range [@start, @end) have attributes
* such that the bits in @mask match @attrs.
@@ -2648,7 +2665,24 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm,
return kvm_vm_set_mem_attributes(kvm, start, end, attrs->attributes);
}
+#else /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
+static unsigned long kvm_get_vm_memory_attributes(struct kvm *kvm, gfn_t gfn)
+{
+ BUILD_BUG_ON(1);
+}
#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
+static void kvm_init_memory_attributes(void)
+{
+ if (vm_memory_attributes)
+ static_call_update(__kvm_get_memory_attributes,
+ kvm_get_vm_memory_attributes);
+ else
+ static_call_update(__kvm_get_memory_attributes,
+ (void *)__static_call_return0);
+}
+#else /* CONFIG_KVM_MEMORY_ATTRIBUTES */
+static void kvm_init_memory_attributes(void) { }
+#endif /* CONFIG_KVM_MEMORY_ATTRIBUTES */
struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn)
{
@@ -4947,6 +4981,9 @@ static int kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)
return 1;
#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
case KVM_CAP_MEMORY_ATTRIBUTES:
+ if (!vm_memory_attributes)
+ return 0;
+
return kvm_supported_mem_attributes(kvm);
#endif
#ifdef CONFIG_KVM_GUEST_MEMFD
@@ -5353,6 +5390,10 @@ static long kvm_vm_ioctl(struct file *filp,
case KVM_SET_MEMORY_ATTRIBUTES: {
struct kvm_memory_attributes attrs;
+ r = -ENOTTY;
+ if (!vm_memory_attributes)
+ goto out;
+
r = -EFAULT;
if (copy_from_user(&attrs, argp, sizeof(attrs)))
goto out;
@@ -6539,6 +6580,7 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align, struct module *module)
kvm_preempt_ops.sched_in = kvm_sched_in;
kvm_preempt_ops.sched_out = kvm_sched_out;
+ kvm_init_memory_attributes();
kvm_init_debug();
r = kvm_vfio_ops_init();
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 03/37] KVM: Enumerate support for PRIVATE memory iff kvm_arch_has_private_mem is defined
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
From: Sean Christopherson <seanjc@google.com>
Explicitly guard reporting support for KVM_MEMORY_ATTRIBUTE_PRIVATE based
on kvm_arch_has_private_mem being #defined in anticipation of decoupling
kvm_supported_mem_attributes() from CONFIG_KVM_VM_MEMORY_ATTRIBUTES.
guest_memfd support for memory attributes will be unconditional to avoid
yet more macros (all architectures that support guest_memfd are expected to
use per-gmem attributes at some point), at which point enumerating support
KVM_MEMORY_ATTRIBUTE_PRIVATE based solely on memory attributes being
supported _somewhere_ would result in KVM over-reporting support on arm64.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
include/linux/kvm_host.h | 2 +-
virt/kvm/kvm_main.c | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 66a5e05bb5b7..af2fcfff7692 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -721,7 +721,7 @@ static inline int kvm_arch_vcpu_memslots_id(struct kvm_vcpu *vcpu)
}
#endif
-#ifndef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
+#ifndef kvm_arch_has_private_mem
static inline bool kvm_arch_has_private_mem(struct kvm *kvm)
{
return false;
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 11c34311b0ac..26e0d532ba03 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2444,8 +2444,10 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm,
#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
static u64 kvm_supported_mem_attributes(struct kvm *kvm)
{
+#ifdef kvm_arch_has_private_mem
if (!kvm || kvm_arch_has_private_mem(kvm))
return KVM_MEMORY_ATTRIBUTE_PRIVATE;
+#endif
return 0;
}
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 02/37] KVM: Rename KVM_GENERIC_MEMORY_ATTRIBUTES to KVM_VM_MEMORY_ATTRIBUTES
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
From: Sean Christopherson <seanjc@google.com>
Rename the per-VM memory attributes Kconfig to make it explicitly about
per-VM attributes in anticipation of adding memory attributes support to
guest_memfd, at which point it will be possible (and desirable) to have
memory attributes without the per-VM support, even in x86.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/Kconfig | 6 +++---
arch/x86/kvm/mmu/mmu.c | 2 +-
arch/x86/kvm/x86.c | 2 +-
include/linux/kvm_host.h | 8 ++++----
include/trace/events/kvm.h | 4 ++--
virt/kvm/Kconfig | 2 +-
virt/kvm/kvm_main.c | 14 +++++++-------
8 files changed, 20 insertions(+), 20 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 5a3bfa293e8b..d427cf9187c3 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -2307,7 +2307,7 @@ void kvm_configure_mmu(bool enable_tdp, int tdp_forced_root_level,
int tdp_max_root_level, int tdp_huge_page_level);
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
#define kvm_arch_has_private_mem(kvm) ((kvm)->arch.has_private_mem)
#endif
diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
index 278f08194ec8..1683dbab870e 100644
--- a/arch/x86/kvm/Kconfig
+++ b/arch/x86/kvm/Kconfig
@@ -84,7 +84,7 @@ config KVM_SW_PROTECTED_VM
bool "Enable support for KVM software-protected VMs"
depends on EXPERT
depends on KVM_X86 && X86_64
- select KVM_GENERIC_MEMORY_ATTRIBUTES
+ select KVM_VM_MEMORY_ATTRIBUTES
help
Enable support for KVM software-protected VMs. Currently, software-
protected VMs are purely a development and testing vehicle for
@@ -135,7 +135,7 @@ config KVM_INTEL_TDX
bool "Intel Trust Domain Extensions (TDX) support"
default y
depends on INTEL_TDX_HOST
- select KVM_GENERIC_MEMORY_ATTRIBUTES
+ select KVM_VM_MEMORY_ATTRIBUTES
select HAVE_KVM_ARCH_GMEM_POPULATE
help
Provides support for launching Intel Trust Domain Extensions (TDX)
@@ -159,7 +159,7 @@ config KVM_AMD_SEV
depends on KVM_AMD && X86_64
depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m)
select ARCH_HAS_CC_PLATFORM
- select KVM_GENERIC_MEMORY_ATTRIBUTES
+ select KVM_VM_MEMORY_ATTRIBUTES
select HAVE_KVM_ARCH_GMEM_PREPARE
select HAVE_KVM_ARCH_GMEM_INVALIDATE
select HAVE_KVM_ARCH_GMEM_POPULATE
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 02c450686b4a..c7de8ff84fd2 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -7890,7 +7890,7 @@ void kvm_mmu_pre_destroy_vm(struct kvm *kvm)
vhost_task_stop(kvm->arch.nx_huge_page_recovery_thread);
}
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
static bool hugepage_test_mixed(struct kvm_memory_slot *slot, gfn_t gfn,
int level)
{
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index ff8812f3a129..b2e93f836dca 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -13437,7 +13437,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *kvm,
}
}
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
kvm_mmu_init_memslot_memory_attributes(kvm, slot);
#endif
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index d93f75b05ae2..66a5e05bb5b7 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -721,7 +721,7 @@ static inline int kvm_arch_vcpu_memslots_id(struct kvm_vcpu *vcpu)
}
#endif
-#ifndef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifndef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
static inline bool kvm_arch_has_private_mem(struct kvm *kvm)
{
return false;
@@ -871,7 +871,7 @@ struct kvm {
#ifdef CONFIG_HAVE_KVM_PM_NOTIFIER
struct notifier_block pm_notifier;
#endif
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
/* Protected by slots_lock (for writes) and RCU (for reads) */
struct xarray mem_attr_array;
#endif
@@ -2515,7 +2515,7 @@ static inline bool kvm_memslot_is_gmem_only(const struct kvm_memory_slot *slot)
return slot->flags & KVM_MEMSLOT_GMEM_ONLY;
}
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn_t gfn)
{
return xa_to_value(xa_load(&kvm->mem_attr_array, gfn));
@@ -2537,7 +2537,7 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn)
{
return false;
}
-#endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */
+#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
#ifdef CONFIG_KVM_GUEST_MEMFD
int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h
index b282e3a86769..1ba72bd73ea2 100644
--- a/include/trace/events/kvm.h
+++ b/include/trace/events/kvm.h
@@ -358,7 +358,7 @@ TRACE_EVENT(kvm_dirty_ring_exit,
TP_printk("vcpu %d", __entry->vcpu_id)
);
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
/*
* @start: Starting address of guest memory range
* @end: End address of guest memory range
@@ -383,7 +383,7 @@ TRACE_EVENT(kvm_vm_set_mem_attributes,
TP_printk("%#016llx -- %#016llx [0x%lx]",
__entry->start, __entry->end, __entry->attr)
);
-#endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */
+#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
TRACE_EVENT(kvm_unmap_hva_range,
TP_PROTO(unsigned long start, unsigned long end),
diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
index 267c7369c765..c12dc19f0a5e 100644
--- a/virt/kvm/Kconfig
+++ b/virt/kvm/Kconfig
@@ -105,7 +105,7 @@ config KVM_MMU_LOCKLESS_AGING
depends on KVM_GENERIC_MMU_NOTIFIER
bool
-config KVM_GENERIC_MEMORY_ATTRIBUTES
+config KVM_VM_MEMORY_ATTRIBUTES
depends on KVM_GENERIC_MMU_NOTIFIER
bool
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 5b5b69c97665..11c34311b0ac 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1132,7 +1132,7 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname)
spin_lock_init(&kvm->mn_invalidate_lock);
rcuwait_init(&kvm->mn_memslots_update_rcuwait);
xa_init(&kvm->vcpu_array);
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
xa_init(&kvm->mem_attr_array);
#endif
@@ -1323,7 +1323,7 @@ static void kvm_destroy_vm(struct kvm *kvm)
cleanup_srcu_struct(&kvm->irq_srcu);
srcu_barrier(&kvm->srcu);
cleanup_srcu_struct(&kvm->srcu);
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
xa_destroy(&kvm->mem_attr_array);
#endif
kvm_arch_free_vm(kvm);
@@ -2441,7 +2441,7 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm,
}
#endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
static u64 kvm_supported_mem_attributes(struct kvm *kvm)
{
if (!kvm || kvm_arch_has_private_mem(kvm))
@@ -2646,7 +2646,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm,
return kvm_vm_set_mem_attributes(kvm, start, end, attrs->attributes);
}
-#endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */
+#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn)
{
@@ -4943,7 +4943,7 @@ static int kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)
case KVM_CAP_SYSTEM_EVENT_DATA:
case KVM_CAP_DEVICE_CTRL:
return 1;
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
case KVM_CAP_MEMORY_ATTRIBUTES:
return kvm_supported_mem_attributes(kvm);
#endif
@@ -5347,7 +5347,7 @@ static long kvm_vm_ioctl(struct file *filp,
break;
}
#endif /* CONFIG_HAVE_KVM_IRQ_ROUTING */
-#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
case KVM_SET_MEMORY_ATTRIBUTES: {
struct kvm_memory_attributes attrs;
@@ -5358,7 +5358,7 @@ static long kvm_vm_ioctl(struct file *filp,
r = kvm_vm_ioctl_set_mem_attributes(kvm, &attrs);
break;
}
-#endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */
+#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
case KVM_CREATE_DEVICE: {
struct kvm_create_device cd;
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 01/37] KVM: guest_memfd: Introduce per-gmem attributes, use to guard user mappings
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao, Ackerley Tng
In-Reply-To: <cover.1770071243.git.ackerleytng@google.com>
From: Sean Christopherson <seanjc@google.com>
Start plumbing in guest_memfd support for in-place private<=>shared
conversions by tracking attributes via a maple tree. KVM currently tracks
private vs. shared attributes on a per-VM basis, which made sense when a
guest_memfd _only_ supported private memory, but tracking per-VM simply
can't work for in-place conversions as the shareability of a given page
needs to be per-gmem_inode, not per-VM.
Use the filemap invalidation lock to protect the maple tree, as taking the
lock for read when faulting in memory (for userspace or the guest) isn't
expected to result in meaningful contention, and using a separate lock
would add significant complexity (avoid deadlock is quite difficult).
Signed-off-by: Sean Christopherson <seanjc@google.com>
Co-developed-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Co-developed-by: Vishal Annapurve <vannapurve@google.com>
Signed-off-by: Vishal Annapurve <vannapurve@google.com>
Co-developed-by: Fuad Tabba <tabba@google.com>
Signed-off-by: Fuad Tabba <tabba@google.com>
---
virt/kvm/guest_memfd.c | 134 ++++++++++++++++++++++++++++++++++++-----
1 file changed, 118 insertions(+), 16 deletions(-)
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index fdaea3422c30..7a970e5fab67 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -4,6 +4,7 @@
#include <linux/falloc.h>
#include <linux/fs.h>
#include <linux/kvm_host.h>
+#include <linux/maple_tree.h>
#include <linux/mempolicy.h>
#include <linux/pseudo_fs.h>
#include <linux/pagemap.h>
@@ -32,6 +33,7 @@ struct gmem_inode {
struct inode vfs_inode;
u64 flags;
+ struct maple_tree attributes;
};
static __always_inline struct gmem_inode *GMEM_I(struct inode *inode)
@@ -59,6 +61,31 @@ static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn)
return gfn - slot->base_gfn + slot->gmem.pgoff;
}
+static u64 kvm_gmem_get_attributes(struct inode *inode, pgoff_t index)
+{
+ struct maple_tree *mt = &GMEM_I(inode)->attributes;
+ void *entry = mtree_load(mt, index);
+
+ /*
+ * The lock _must_ be held for lookups, as some maple tree operations,
+ * e.g. append, are unsafe (return inaccurate information) with respect
+ * to concurrent RCU-protected lookups.
+ */
+ lockdep_assert(mt_lock_is_held(mt));
+
+ return WARN_ON_ONCE(!entry) ? 0 : xa_to_value(entry);
+}
+
+static bool kvm_gmem_is_private_mem(struct inode *inode, pgoff_t index)
+{
+ return kvm_gmem_get_attributes(inode, index) & KVM_MEMORY_ATTRIBUTE_PRIVATE;
+}
+
+static bool kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index)
+{
+ return !kvm_gmem_is_private_mem(inode, index);
+}
+
static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,
pgoff_t index, struct folio *folio)
{
@@ -402,10 +429,13 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)
if (((loff_t)vmf->pgoff << PAGE_SHIFT) >= i_size_read(inode))
return VM_FAULT_SIGBUS;
- if (!(GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED))
- return VM_FAULT_SIGBUS;
+ filemap_invalidate_lock_shared(inode->i_mapping);
+ if (kvm_gmem_is_shared_mem(inode, vmf->pgoff))
+ folio = kvm_gmem_get_folio(inode, vmf->pgoff);
+ else
+ folio = ERR_PTR(-EACCES);
+ filemap_invalidate_unlock_shared(inode->i_mapping);
- folio = kvm_gmem_get_folio(inode, vmf->pgoff);
if (IS_ERR(folio)) {
if (PTR_ERR(folio) == -EAGAIN)
return VM_FAULT_RETRY;
@@ -561,6 +591,51 @@ bool __weak kvm_arch_supports_gmem_init_shared(struct kvm *kvm)
return true;
}
+static int kvm_gmem_init_inode(struct inode *inode, loff_t size, u64 flags)
+{
+ struct gmem_inode *gi = GMEM_I(inode);
+ MA_STATE(mas, &gi->attributes, 0, (size >> PAGE_SHIFT) - 1);
+ u64 attrs;
+ int r;
+
+ inode->i_op = &kvm_gmem_iops;
+ inode->i_mapping->a_ops = &kvm_gmem_aops;
+ inode->i_mode |= S_IFREG;
+ inode->i_size = size;
+ mapping_set_gfp_mask(inode->i_mapping, GFP_HIGHUSER);
+
+ /*
+ * guest_memfd memory is neither migratable nor swappable: set
+ * inaccessible to gate off both.
+ */
+ mapping_set_inaccessible(inode->i_mapping);
+ WARN_ON_ONCE(!mapping_unevictable(inode->i_mapping));
+
+ gi->flags = flags;
+
+ mt_set_external_lock(&gi->attributes,
+ &inode->i_mapping->invalidate_lock);
+
+ /*
+ * Store default attributes for the entire gmem instance. Ensuring every
+ * index is represented in the maple tree at all times simplifies the
+ * conversion and merging logic.
+ */
+ attrs = gi->flags & GUEST_MEMFD_FLAG_INIT_SHARED ? 0 : KVM_MEMORY_ATTRIBUTE_PRIVATE;
+
+ /*
+ * Acquire the invalidation lock purely to make lockdep happy. The
+ * maple tree library expects all stores to be protected via the lock,
+ * and the library can't know when the tree is reachable only by the
+ * caller, as is the case here.
+ */
+ filemap_invalidate_lock(inode->i_mapping);
+ r = mas_store_gfp(&mas, xa_mk_value(attrs), GFP_KERNEL);
+ filemap_invalidate_unlock(inode->i_mapping);
+
+ return r;
+}
+
static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
{
static const char *name = "[kvm-gmem]";
@@ -591,16 +666,9 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
goto err_fops;
}
- inode->i_op = &kvm_gmem_iops;
- inode->i_mapping->a_ops = &kvm_gmem_aops;
- inode->i_mode |= S_IFREG;
- inode->i_size = size;
- mapping_set_gfp_mask(inode->i_mapping, GFP_HIGHUSER);
- mapping_set_inaccessible(inode->i_mapping);
- /* Unmovable mappings are supposed to be marked unevictable as well. */
- WARN_ON_ONCE(!mapping_unevictable(inode->i_mapping));
-
- GMEM_I(inode)->flags = flags;
+ err = kvm_gmem_init_inode(inode, size, flags);
+ if (err)
+ goto err_inode;
file = alloc_file_pseudo(inode, kvm_gmem_mnt, name, O_RDWR, &kvm_gmem_fops);
if (IS_ERR(file)) {
@@ -804,9 +872,13 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
if (!file)
return -EFAULT;
+ filemap_invalidate_lock_shared(file_inode(file)->i_mapping);
+
folio = __kvm_gmem_get_pfn(file, slot, index, pfn, &is_prepared, max_order);
- if (IS_ERR(folio))
- return PTR_ERR(folio);
+ if (IS_ERR(folio)) {
+ r = PTR_ERR(folio);
+ goto out;
+ }
if (!is_prepared)
r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio);
@@ -818,6 +890,8 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
else
folio_put(folio);
+out:
+ filemap_invalidate_unlock_shared(file_inode(file)->i_mapping);
return r;
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_get_pfn);
@@ -931,13 +1005,41 @@ static struct inode *kvm_gmem_alloc_inode(struct super_block *sb)
mpol_shared_policy_init(&gi->policy, NULL);
+ /*
+ * Memory attributes are protected by the filemap invalidation lock, but
+ * the lock structure isn't available at this time. Immediately mark
+ * maple tree as using external locking so that accessing the tree
+ * before it's fully initialized results in NULL pointer dereferences
+ * and not more subtle bugs.
+ */
+ mt_init_flags(&gi->attributes, MT_FLAGS_LOCK_EXTERN);
+
gi->flags = 0;
return &gi->vfs_inode;
}
static void kvm_gmem_destroy_inode(struct inode *inode)
{
- mpol_free_shared_policy(&GMEM_I(inode)->policy);
+ struct gmem_inode *gi = GMEM_I(inode);
+
+ mpol_free_shared_policy(&gi->policy);
+
+ /*
+ * Note! Checking for an empty tree is functionally necessary
+ * to avoid explosions if the tree hasn't been fully
+ * initialized, i.e. if the inode is being destroyed before
+ * guest_memfd can set the external lock, lockdep would find
+ * that the tree's internal ma_lock was not held.
+ */
+ if (!mtree_empty(&gi->attributes)) {
+ /*
+ * Acquire the invalidation lock purely to make lockdep happy,
+ * the inode is unreachable at this point.
+ */
+ filemap_invalidate_lock(inode->i_mapping);
+ __mt_destroy(&gi->attributes);
+ filemap_invalidate_unlock(inode->i_mapping);
+ }
}
static void kvm_gmem_free_inode(struct inode *inode)
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply related
* [RFC PATCH v2 00/37] guest_memfd: In-place conversion support
From: Ackerley Tng @ 2026-02-02 22:29 UTC (permalink / raw)
To: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86
Cc: aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tabba, tglx, vannapurve, vbabka, willy, wyihan,
yan.y.zhao, Ackerley Tng
Here's a second revision of guest_memfd In-place conversion support.
In this version, other than addressing comments from RFCv1 [1], the largest
change is that guest_memfd now does not avoid participation in LRU; it
participates in LRU by joining the unevictable list (no change from before this
series).
While checking for elevated refcounts during shared to private conversions,
guest_memfd will now do an lru_add_drain_all() if elevated refcounts were found,
before concluding that there are true users of the shared folio and erroring
out.
I'd still like feedback on these points, if any:
1. Having private/shared status stored in a maple tree (Thanks Michael for your
support of using maple trees over xarrays for performance! [5]).
2. Having a new guest_memfd ioctl (not a vm ioctl) that performs conversions.
3. Using ioctls/structs/input attribute similar to the existing vm ioctl
KVM_SET_MEMORY_ATTRIBUTES to perform conversions.
4. Storing requested attributes directly in the maple tree.
5. Using a KVM module-wide param to toggle between setting memory attributes via
vm and guest_memfd ioctls (making them mututally exclusive - a single loaded
KVM module can only do one of the two.).
This series is based on kvm/next as at 2026-01-21, and here's the tree for your
convenience:
https://github.com/googleprodkernel/linux-cc/commits/guest_memfd-inplace-conversion-v2
The "Don't set FGP_ACCESSED when getting folios" patch from RFCv1 is still
useful but no longer related to conversion, and was posted separately [6].
Older series:
+ RFCv1 is at [1]
+ Previous versions of this feature, part of other series, are available at
[2][3][4].
[1] https://lore.kernel.org/all/cover.1760731772.git.ackerleytng@google.com/T/
[2] https://lore.kernel.org/all/bd163de3118b626d1005aa88e71ef2fb72f0be0f.1726009989.git.ackerleytng@google.com/
[3] https://lore.kernel.org/all/20250117163001.2326672-6-tabba@google.com/
[4] https://lore.kernel.org/all/b784326e9ccae6a08388f1bf39db70a2204bdc51.1747264138.git.ackerleytng@google.com/
[5] https://lore.kernel.org/all/20250529054227.hh2f4jmyqf6igd3i@amd.com/
[6] https://lore.kernel.org/all/20260129172646.2361462-1-ackerleytng@google.com/
Ackerley Tng (19):
KVM: guest_memfd: Update kvm_gmem_populate() to use gmem attributes
KVM: Introduce KVM_SET_MEMORY_ATTRIBUTES2
KVM: guest_memfd: Add support for KVM_SET_MEMORY_ATTRIBUTES2
KVM: guest_memfd: Handle lru_add fbatch refcounts during conversion
safety check
KVM: selftests: Update framework to use KVM_SET_MEMORY_ATTRIBUTES2
KVM: selftests: Test using guest_memfd for guest private memory
KVM: selftests: Test basic single-page conversion flow
KVM: selftests: Test conversion flow when INIT_SHARED
KVM: selftests: Test indexing in guest_memfd
KVM: selftests: Test conversion before allocation
KVM: selftests: Convert with allocated folios in different layouts
KVM: selftests: Test precision of conversion
KVM: selftests: Test that truncation does not change shared/private
status
KVM: selftests: Test conversion with elevated page refcount
KVM: selftests: Reset shared memory after hole-punching
KVM: selftests: Provide function to look up guest_memfd details from
gpa
KVM: selftests: Make TEST_EXPECT_SIGBUS thread-safe
KVM: selftests: Update private_mem_conversions_test to mmap()
guest_memfd
KVM: selftests: Add script to exercise private_mem_conversions_test
Sean Christopherson (18):
KVM: guest_memfd: Introduce per-gmem attributes, use to guard user
mappings
KVM: Rename KVM_GENERIC_MEMORY_ATTRIBUTES to KVM_VM_MEMORY_ATTRIBUTES
KVM: Enumerate support for PRIVATE memory iff kvm_arch_has_private_mem
is defined
KVM: Stub in ability to disable per-VM memory attribute tracking
KVM: guest_memfd: Wire up kvm_get_memory_attributes() to per-gmem
attributes
KVM: guest_memfd: Enable INIT_SHARED on guest_memfd for x86 Coco VMs
KVM: Move KVM_VM_MEMORY_ATTRIBUTES config definition to x86
KVM: Let userspace disable per-VM mem attributes, enable per-gmem
attributes
KVM: selftests: Create gmem fd before "regular" fd when adding memslot
KVM: selftests: Rename guest_memfd{,_offset} to gmem_{fd,offset}
KVM: selftests: Add support for mmap() on guest_memfd in core library
KVM: selftests: Add selftests global for guest memory attributes
capability
KVM: selftests: Add helpers for calling ioctls on guest_memfd
KVM: selftests: Test that shared/private status is consistent across
processes
KVM: selftests: Provide common function to set memory attributes
KVM: selftests: Check fd/flags provided to mmap() when setting up
memslot
KVM: selftests: Update pre-fault test to work with per-guest_memfd
attributes
KVM: selftests: Update private memory exits test work with per-gmem
attributes
Documentation/virt/kvm/api.rst | 72 ++-
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/Kconfig | 15 +-
arch/x86/kvm/mmu/mmu.c | 4 +-
arch/x86/kvm/x86.c | 13 +-
include/linux/kvm_host.h | 53 +-
include/trace/events/kvm.h | 4 +-
include/uapi/linux/kvm.h | 17 +
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile.kvm | 1 +
.../kvm/guest_memfd_conversions_test.c | 486 ++++++++++++++++++
.../testing/selftests/kvm/guest_memfd_test.c | 57 +-
.../testing/selftests/kvm/include/kvm_util.h | 128 ++++-
.../testing/selftests/kvm/include/test_util.h | 31 +-
tools/testing/selftests/kvm/lib/kvm_util.c | 130 +++--
tools/testing/selftests/kvm/lib/test_util.c | 7 -
.../selftests/kvm/pre_fault_memory_test.c | 2 +-
.../kvm/x86/private_mem_conversions_test.c | 48 +-
.../kvm/x86/private_mem_conversions_test.py | 152 ++++++
.../kvm/x86/private_mem_kvm_exits_test.c | 36 +-
virt/kvm/Kconfig | 4 +-
virt/kvm/guest_memfd.c | 399 +++++++++++++-
virt/kvm/kvm_main.c | 104 +++-
23 files changed, 1590 insertions(+), 176 deletions(-)
create mode 100644 tools/testing/selftests/kvm/guest_memfd_conversions_test.c
create mode 100755 tools/testing/selftests/kvm/x86/private_mem_conversions_test.py
--
2.53.0.rc1.225.gd81095ad13-goog
^ permalink raw reply
* [PATCH AUTOSEL 6.18] tracing: Avoid possible signed 64-bit truncation
From: Sasha Levin @ 2026-02-02 21:46 UTC (permalink / raw)
To: patches, stable
Cc: Ian Rogers, Mathieu Desnoyers, Masami Hiramatsu (Google),
Steven Rostedt (Google), Sasha Levin, linux-kernel,
linux-trace-kernel
In-Reply-To: <20260202214643.212290-1-sashal@kernel.org>
From: Ian Rogers <irogers@google.com>
[ Upstream commit 00f13e28a9c3acd40f0551cde7e9d2d1a41585bf ]
64-bit truncation to 32-bit can result in the sign of the truncated
value changing. The cmp_mod_entry is used in bsearch and so the
truncation could result in an invalid search order. This would only
happen were the addresses more than 2GB apart and so unlikely, but
let's fix the potentially broken compare anyway.
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://patch.msgid.link/20260108002625.333331-1-irogers@google.com
Signed-off-by: Ian Rogers <irogers@google.com>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
LLM Generated explanations, may be completely bogus:
The original buggy code was introduced in v6.15 and later. It's only
present in 6.15+ kernels.
### 8. SUMMARY OF ANALYSIS
**The Bug:**
- The `cmp_mod_entry()` function uses subtraction of two `unsigned long`
values and returns the result as `int`
- On 64-bit systems, if addresses differ by more than 2^31 (~2GB), the
truncation from 64-bit to 32-bit can flip the sign
- This would cause `bsearch()` to make wrong decisions about search
direction
- Result: potentially incorrect module address lookups in trace data
**The Fix:**
- Replaces arithmetic subtraction with simple comparisons
- Returns -1, 0, or 1 directly based on comparisons
- No overflow or truncation possible with the new code
- Logic is more readable and provably correct
**Stable Criteria Evaluation:**
1. ✅ **Obviously correct and tested**: Simple logic, reviewed by
maintainers
2. ✅ **Fixes a real bug**: Yes, a potential comparator correctness bug
3. ⚠️ **Important issue**: Moderate - unlikely to trigger (requires >2GB
address separation) but could cause incorrect trace output
4. ✅ **Small and contained**: Only changes one function body (~6 lines)
5. ✅ **No new features**: Pure bug fix
6. ✅ **Applies cleanly**: Should apply to 6.15+ kernels where this code
exists
**Risk Assessment:**
- Very low risk - the change is small and the new logic is simpler
- The original code has a provable bug (integer overflow on truncation)
- The new code has no such issues
**Concerns:**
- The code only exists in 6.15+ kernels (introduced March 2025)
- The bug is "unlikely" per the author (requires addresses >2GB apart)
- No known real-world reports of this actually causing issues
### DECISION
This is a valid bug fix that:
- Fixes a real (though unlikely to trigger) bug in the comparator
function
- Is very small and self-contained
- Has been reviewed and acked by the tracing maintainers
- Has near-zero regression risk
- Applies to 6.15+ kernels only
The fix is surgical, obviously correct, and addresses a potential
correctness issue. While the bug is unlikely to trigger in practice
(addresses must be >2GB apart), it could cause silent data corruption in
trace output when it does trigger. The fix is trivial and risk-free.
**YES**
kernel/trace/trace.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 142e3b737f0bc..907923d5f8bbb 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -6061,10 +6061,10 @@ static int cmp_mod_entry(const void *key, const void *pivot)
unsigned long addr = (unsigned long)key;
const struct trace_mod_entry *ent = pivot;
- if (addr >= ent[0].mod_addr && addr < ent[1].mod_addr)
- return 0;
- else
- return addr - ent->mod_addr;
+ if (addr < ent[0].mod_addr)
+ return -1;
+
+ return addr >= ent[1].mod_addr;
}
/**
--
2.51.0
^ permalink raw reply related
* Re: [PATCH bpf-next] ftrace: Fix direct_functions leak in update_ftrace_direct_del
From: kernel test robot @ 2026-02-02 18:51 UTC (permalink / raw)
To: Jiri Olsa, Steven Rostedt, Alexei Starovoitov
Cc: oe-kbuild-all, bpf, linux-kernel, linux-trace-kernel,
Daniel Borkmann, Andrii Nakryiko, Menglong Dong, Song Liu
In-Reply-To: <20260202075849.1684369-1-jolsa@kernel.org>
Hi Jiri,
kernel test robot noticed the following build warnings:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Jiri-Olsa/ftrace-Fix-direct_functions-leak-in-update_ftrace_direct_del/20260202-160850
base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link: https://lore.kernel.org/r/20260202075849.1684369-1-jolsa%40kernel.org
patch subject: [PATCH bpf-next] ftrace: Fix direct_functions leak in update_ftrace_direct_del
config: i386-randconfig-r123-20260202 (https://download.01.org/0day-ci/archive/20260203/202602030237.Ze7K6UoS-lkp@intel.com/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260203/202602030237.Ze7K6UoS-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202602030237.Ze7K6UoS-lkp@intel.com/
sparse warnings: (new ones prefixed by >>)
kernel/trace/ftrace.c:1094:43: sparse: got struct ftrace_hash *
kernel/trace/ftrace.c:1326:40: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:1326:40: sparse: expected struct ftrace_hash *hash
kernel/trace/ftrace.c:1326:40: sparse: got struct ftrace_hash [noderef] __rcu *filter_hash
kernel/trace/ftrace.c:1327:40: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:1327:40: sparse: expected struct ftrace_hash *hash
kernel/trace/ftrace.c:1327:40: sparse: got struct ftrace_hash [noderef] __rcu *notrace_hash
kernel/trace/ftrace.c:1328:37: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:1328:37: sparse: expected struct ftrace_hash [noderef] __rcu *filter_hash
kernel/trace/ftrace.c:1328:37: sparse: got struct ftrace_hash *
kernel/trace/ftrace.c:1329:38: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *notrace_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:1329:38: sparse: expected struct ftrace_hash [noderef] __rcu *notrace_hash
kernel/trace/ftrace.c:1329:38: sparse: got struct ftrace_hash *
kernel/trace/ftrace.c:2141:54: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected struct ftrace_hash *old_hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:2141:54: sparse: expected struct ftrace_hash *old_hash
kernel/trace/ftrace.c:2141:54: sparse: got struct ftrace_hash [noderef] __rcu *filter_hash
kernel/trace/ftrace.c:1533:9: sparse: sparse: incompatible types in comparison expression (different address spaces):
kernel/trace/ftrace.c:1533:9: sparse: struct ftrace_hash [noderef] __rcu *
kernel/trace/ftrace.c:1533:9: sparse: struct ftrace_hash *
kernel/trace/ftrace.c:1549:39: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:1550:40: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:1551:40: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:1552:42: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:1723:18: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_ops *ops @@ got struct ftrace_ops [noderef] __rcu *[addressable] [toplevel] ftrace_ops_list @@
kernel/trace/ftrace.c:1724:43: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_ops *ops @@ got struct ftrace_ops [noderef] __rcu *next @@
kernel/trace/ftrace.c:1785:14: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:1786:22: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *notrace_hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:2119:50: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:2130:50: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:2613:53: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *static [toplevel] direct_functions @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:2624:36: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *static [toplevel] direct_functions @@
kernel/trace/ftrace.c:3425:51: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *B @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3426:66: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:3432:52: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *B @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3433:66: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:3446:41: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3447:51: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *src @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3450:52: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *notrace_hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3454:52: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *src @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3469:39: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3470:42: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3478:17: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3484:81: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3488:54: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *notrace_hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3490:56: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *new_hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3520:60: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *new_hash1 @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3521:49: sparse: sparse: incorrect type in argument 3 (different address spaces) @@ expected struct ftrace_hash *new_hash2 @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3560:45: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3562:46: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *notrace_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3564:48: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3566:49: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *notrace_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3572:17: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3573:17: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3579:34: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *save_filter_hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3580:35: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *save_notrace_hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3582:45: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash *[addressable] filter_hash @@
kernel/trace/ftrace.c:3583:46: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *notrace_hash @@ got struct ftrace_hash *[addressable] notrace_hash @@
kernel/trace/ftrace.c:3588:53: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash *save_filter_hash @@
kernel/trace/ftrace.c:3589:54: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *notrace_hash @@ got struct ftrace_hash *save_notrace_hash @@
kernel/trace/ftrace.c:3636:31: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3637:32: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *notrace_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3652:59: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *[addressable] filter_hash @@
kernel/trace/ftrace.c:3653:59: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *[addressable] notrace_hash @@
kernel/trace/ftrace.c:3658:43: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *[addressable] filter_hash @@
kernel/trace/ftrace.c:3659:43: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *[addressable] notrace_hash @@
kernel/trace/ftrace.c:3661:39: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *[addressable] filter_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3662:40: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *[addressable] notrace_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3704:48: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3705:48: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3706:45: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3707:46: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *notrace_hash @@ got struct ftrace_hash * @@
kernel/trace/ftrace.c:3993:14: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:4010:22: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:4699:22: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:4702:22: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:5113:27: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:5115:27: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:5495:19: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:5639:19: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:5645:34: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash *[assigned] old_hash @@
kernel/trace/ftrace.c:5910:27: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:5912:27: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:5993:50: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *static [toplevel] direct_functions @@
kernel/trace/ftrace.c:5996:51: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *static [toplevel] direct_functions @@
kernel/trace/ftrace.c:6012:50: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:6056:14: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:6106:19: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *free_hash @@ got struct ftrace_hash [noderef] __rcu *static [toplevel] direct_functions @@
kernel/trace/ftrace.c:6171:50: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:6209:52: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *static [addressable] [assigned] [toplevel] direct_functions @@
kernel/trace/ftrace.c:6357:48: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *static [addressable] [assigned] [toplevel] direct_functions @@
kernel/trace/ftrace.c:6362:25: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *old_filter_hash @@ got struct ftrace_hash [noderef] __rcu * @@
kernel/trace/ftrace.c:6378:41: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *a @@ got struct ftrace_hash [noderef] __rcu *static [addressable] [assigned] [toplevel] direct_functions @@
kernel/trace/ftrace.c:6382:30: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *old_direct_functions @@ got struct ftrace_hash [noderef] __rcu *static [addressable] [assigned] [toplevel] direct_functions @@
kernel/trace/ftrace.c:6389:45: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash *[assigned] new_filter_hash @@
kernel/trace/ftrace.c:6394:53: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash [noderef] __rcu *filter_hash @@ got struct ftrace_hash *old_filter_hash @@
kernel/trace/ftrace.c:6489:30: sparse: sparse: incompatible types in comparison expression (different address spaces):
kernel/trace/ftrace.c:6489:30: sparse: struct ftrace_hash [noderef] __rcu *
kernel/trace/ftrace.c:6489:30: sparse: struct ftrace_hash *
kernel/trace/ftrace.c:6503:50: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *static [addressable] [assigned] [toplevel] direct_functions @@
kernel/trace/ftrace.c:6514:41: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *a @@ got struct ftrace_hash [noderef] __rcu *static [addressable] [assigned] [toplevel] direct_functions @@
>> kernel/trace/ftrace.c:6540:38: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *[assigned] old_direct_functions @@ got struct ftrace_hash [noderef] __rcu *static [addressable] [assigned] [toplevel] direct_functions @@
kernel/trace/ftrace.c:6589:30: sparse: sparse: incompatible types in comparison expression (different address spaces):
kernel/trace/ftrace.c:6589:30: sparse: struct ftrace_hash [noderef] __rcu *
kernel/trace/ftrace.c:6589:30: sparse: struct ftrace_hash *
kernel/trace/ftrace.c:6969:35: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:6977:35: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash **orig_hash @@ got struct ftrace_hash [noderef] __rcu ** @@
kernel/trace/ftrace.c:7762:46: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:7763:47: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:7767:44: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:7785:18: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_ops *ops @@ got struct ftrace_ops [noderef] __rcu *[addressable] [toplevel] ftrace_ops_list @@
kernel/trace/ftrace.c:7785:66: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_ops *ops @@ got struct ftrace_ops [noderef] __rcu *next @@
kernel/trace/ftrace.c:7837:59: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:7838:59: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:8227:62: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:8228:62: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:8272:36: sparse: sparse: incompatible types in comparison expression (different address spaces):
kernel/trace/ftrace.c:8272:36: sparse: struct ftrace_ops [noderef] __rcu *
kernel/trace/ftrace.c:8272:36: sparse: struct ftrace_ops *
kernel/trace/ftrace.c:9048:14: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:9048:14: sparse: expected struct ftrace_hash *hash
kernel/trace/ftrace.c:9048:14: sparse: got struct ftrace_hash [noderef] __rcu *filter_hash
kernel/trace/ftrace.c:9097:14: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:9097:14: sparse: expected struct ftrace_hash *hash
kernel/trace/ftrace.c:9097:14: sparse: got struct ftrace_hash [noderef] __rcu *filter_hash
kernel/trace/ftrace.c:230:20: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:230:20: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:230:20: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3480:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3480:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3480:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3480:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3480:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3480:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3514:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3514:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3514:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3514:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3514:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3514:29: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:6074:30: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:6083:21: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:6085:17: sparse: sparse: dereference of noderef expression
kernel/trace/ftrace.c:3785:48: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *filter_hash @@
kernel/trace/ftrace.c:3785:48: sparse: expected struct ftrace_hash *hash
kernel/trace/ftrace.c:3785:48: sparse: got struct ftrace_hash [noderef] __rcu *filter_hash
kernel/trace/ftrace.c:3786:49: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct ftrace_hash *hash @@ got struct ftrace_hash [noderef] __rcu *notrace_hash @@
kernel/trace/ftrace.c:3786:49: sparse: expected struct ftrace_hash *hash
kernel/trace/ftrace.c:3786:49: sparse: got struct ftrace_hash [noderef] __rcu *notrace_hash
vim +6540 kernel/trace/ftrace.c
6457
6458 /**
6459 * update_ftrace_direct_del - Updates @ops by removing its direct
6460 * callers provided in @hash
6461 * @ops: The address of the struct ftrace_ops object
6462 * @hash: The address of the struct ftrace_hash object
6463 *
6464 * This is used to delete custom direct callers (ip -> addr) in
6465 * @ops specified via @hash. The @ops will be either unregistered
6466 * updated.
6467 *
6468 * Returns: zero on success. Non zero on error, which includes:
6469 * -EINVAL - The @hash is empty
6470 * -EINVAL - The @ops is not registered
6471 */
6472 int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash)
6473 {
6474 struct ftrace_hash *old_direct_functions = NULL;
6475 struct ftrace_hash *new_direct_functions;
6476 struct ftrace_hash *new_filter_hash = NULL;
6477 struct ftrace_hash *old_filter_hash;
6478 struct ftrace_func_entry *entry;
6479 struct ftrace_func_entry *del;
6480 unsigned long size;
6481 int err = -EINVAL;
6482
6483 if (!hash_count(hash))
6484 return -EINVAL;
6485 if (check_direct_multi(ops))
6486 return -EINVAL;
6487 if (!(ops->flags & FTRACE_OPS_FL_ENABLED))
6488 return -EINVAL;
6489 if (direct_functions == EMPTY_HASH)
6490 return -EINVAL;
6491
6492 mutex_lock(&direct_mutex);
6493
6494 old_filter_hash = ops->func_hash ? ops->func_hash->filter_hash : NULL;
6495
6496 if (!hash_count(old_filter_hash))
6497 goto out_unlock;
6498
6499 /* Make sure requested entries are already registered. */
6500 size = 1 << hash->size_bits;
6501 for (int i = 0; i < size; i++) {
6502 hlist_for_each_entry(entry, &hash->buckets[i], hlist) {
6503 del = __ftrace_lookup_ip(direct_functions, entry->ip);
6504 if (!del || del->direct != entry->direct)
6505 goto out_unlock;
6506 }
6507 }
6508
6509 err = -ENOMEM;
6510 new_filter_hash = hash_sub(old_filter_hash, hash);
6511 if (!new_filter_hash)
6512 goto out_unlock;
6513
6514 new_direct_functions = hash_sub(direct_functions, hash);
6515 if (!new_direct_functions)
6516 goto out_unlock;
6517
6518 /* If there's nothing left, we need to unregister the ops. */
6519 if (ftrace_hash_empty(new_filter_hash)) {
6520 err = unregister_ftrace_function(ops);
6521 if (!err) {
6522 /* cleanup for possible another register call */
6523 ops->func = NULL;
6524 ops->trampoline = 0;
6525 ftrace_free_filter(ops);
6526 ops->func_hash->filter_hash = NULL;
6527 }
6528 } else {
6529 err = ftrace_update_ops(ops, new_filter_hash, EMPTY_HASH);
6530 /*
6531 * new_filter_hash is dup-ed, so we need to release it anyway,
6532 * old_filter_hash either stays on error or is already released
6533 */
6534 }
6535
6536 if (err) {
6537 /* free the new_direct_functions */
6538 old_direct_functions = new_direct_functions;
6539 } else {
> 6540 old_direct_functions = direct_functions;
6541 rcu_assign_pointer(direct_functions, new_direct_functions);
6542 }
6543
6544 out_unlock:
6545 mutex_unlock(&direct_mutex);
6546
6547 if (old_direct_functions && old_direct_functions != EMPTY_HASH)
6548 call_rcu_tasks(&old_direct_functions->rcu, register_ftrace_direct_cb);
6549 free_ftrace_hash(new_filter_hash);
6550
6551 return err;
6552 }
6553
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [PATCH] tracing: remove __printf() attribute on __ftrace_vbprintk()
From: Steven Rostedt @ 2026-02-02 17:59 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Masami Hiramatsu, Simon Horman, Jeff Layton, Anna Schumaker,
Chuck Lever, Arnd Bergmann, Mathieu Desnoyers, Andrew Morton,
Andy Shevchenko, Randy Dunlap, Yury Norov, Joel Fernandes,
linux-kernel, linux-trace-kernel
In-Reply-To: <20260202095834.1328352-1-arnd@kernel.org>
On Mon, 2 Feb 2026 10:58:27 +0100
Arnd Bergmann <arnd@kernel.org> wrote:
> From: Arnd Bergmann <arnd@arndb.de>
>
> The sunrpc change to use trace_printk() for debugging caused
> a new warning for every instance of dprintk() in some configurations,
> when -Wformat-security is enabled:
>
> fs/nfs/getroot.c: In function 'nfs_get_root':
> fs/nfs/getroot.c:90:17: error: format not a string literal and no format arguments [-Werror=format-security]
> 90 | nfs_errorf(fc, "NFS: Couldn't getattr on root");
>
> I've been slowly chipping away at those warnings over time with the
> intention of enabling them by default in the future. While I could not
> figure out why this only happens for this one instance, I see that the
> __trace_bprintk() function is always called with a local variable as
> the format string, rather than a literal.
>
> Remove the __printf(2,3) annotation on this function, as this is can
> only be validated for literals. The format strings still get checked
> because the underlying literal keeps getting passed into __trace_printk()
> in the "else" branch, which is not taken but still evaluated for
> compile-time warnings.
>
> Fixes: ec7d8e68ef0e ("sunrpc: add a Kconfig option to redirect dfprintk() output to trace buffer")
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
-- Steve
> ---
> include/linux/trace_printk.h | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/include/linux/trace_printk.h b/include/linux/trace_printk.h
> index bb5874097f24..2670ec7f4262 100644
> --- a/include/linux/trace_printk.h
> +++ b/include/linux/trace_printk.h
> @@ -107,7 +107,6 @@ do { \
> __trace_printk(_THIS_IP_, fmt, ##args); \
> } while (0)
>
> -extern __printf(2, 3)
> int __trace_bprintk(unsigned long ip, const char *fmt, ...);
>
> extern __printf(2, 3)
^ permalink raw reply
* Re: [PATCH] tracing: remove __printf() attribute on __ftrace_vbprintk()
From: Andy Shevchenko @ 2026-02-02 17:57 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Steven Rostedt, Masami Hiramatsu, Simon Horman, Jeff Layton,
Anna Schumaker, Chuck Lever, Arnd Bergmann, Mathieu Desnoyers,
Andrew Morton, Randy Dunlap, Yury Norov, Joel Fernandes,
linux-kernel, linux-trace-kernel
In-Reply-To: <20260202095834.1328352-1-arnd@kernel.org>
On Mon, Feb 02, 2026 at 10:58:27AM +0100, Arnd Bergmann wrote:
> The sunrpc change to use trace_printk() for debugging caused
> a new warning for every instance of dprintk() in some configurations,
> when -Wformat-security is enabled:
>
> fs/nfs/getroot.c: In function 'nfs_get_root':
> fs/nfs/getroot.c:90:17: error: format not a string literal and no format arguments [-Werror=format-security]
> 90 | nfs_errorf(fc, "NFS: Couldn't getattr on root");
>
> I've been slowly chipping away at those warnings over time with the
> intention of enabling them by default in the future. While I could not
> figure out why this only happens for this one instance, I see that the
> __trace_bprintk() function is always called with a local variable as
> the format string, rather than a literal.
>
> Remove the __printf(2,3) annotation on this function, as this is can
> only be validated for literals. The format strings still get checked
> because the underlying literal keeps getting passed into __trace_printk()
> in the "else" branch, which is not taken but still evaluated for
> compile-time warnings.
Yeah, it was discussed in the past that binary printf() should not be annotated
as we have BPF code that doesn't compile (with the default WERROR=y).
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply
* [PATCH] tracing: Fix ftrace event field alignments
From: Steven Rostedt @ 2026-02-02 16:30 UTC (permalink / raw)
To: LKML, Linux Trace Kernel
Cc: Masami Hiramatsu, Mathieu Desnoyers, jempty.liang
From: Steven Rostedt <rostedt@goodmis.org>
The fields of ftrace specific events (events used to save ftrace internal
events like function traces and trace_printk) are generated similarly to
how normal trace event fields are generated. That is, the fields are added
to a trace_events_fields array that saves the name, offset, size,
alignment and signness of the field. It is used to produce the output in
the format file in tracefs so that tooling knows how to parse the binary
data of the trace events.
The issue is that the macro to determine the alignment of a field used the
alignment of the type when used normally, but not when it is used within a
structure.
This was fixed for normal trace event fields via: 4c3d2f9388d36 ("tracing:
Use a struct alignof to determine trace event field alignment"), but was
missed for the ftrace specific events.
Do the same for ftrace specific events by creating a helper macro:
#define ALIGN_STRUCTFIELD(type) ((int)(__alignof__(struct {type b;})))
and use that as the alignment of types within the ftrace event structures.
Cc: stable@vger.kernel.org
Fixes: 04ae87a52074e ("ftrace: Rework event_create_dir()")
Reported-by: "jempty.liang" <imntjempty@163.com>
Closes: https://lore.kernel.org/all/20260130015740.212343-1-imntjempty@163.com/
Closes: https://lore.kernel.org/all/20260202123342.2544795-1-imntjempty@163.com/
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
kernel/trace/trace_export.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_export.c b/kernel/trace/trace_export.c
index 1698fc22afa0..5b96ac750049 100644
--- a/kernel/trace/trace_export.c
+++ b/kernel/trace/trace_export.c
@@ -14,6 +14,9 @@
#include "trace_output.h"
+/* The alignment of a type when in a structure */
+#define ALIGN_STRUCTFIELD(type) ((int)(__alignof__(struct {type b;})))
+
/* Stub function for events with triggers */
static int ftrace_event_register(struct trace_event_call *call,
enum trace_reg type, void *data)
@@ -88,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void) \
#undef __field_ext
#define __field_ext(_type, _item, _filter_type) { \
.type = #_type, .name = #_item, \
- .size = sizeof(_type), .align = __alignof__(_type), \
+ .size = sizeof(_type), .align = ALIGN_STRUCTFIELD(_type), \
is_signed_type(_type), .filter_type = _filter_type },
--
2.51.0
^ permalink raw reply related
* Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF
From: Steven Rostedt @ 2026-02-02 16:12 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, mhiramat, andrii.nakryiko, linux-kernel, Donglin Peng,
linux-trace-kernel
In-Reply-To: <20260202111548.3555306-3-dolinux.peng@gmail.com>
On Mon, 2 Feb 2026 19:15:48 +0800
Donglin Peng <dolinux.peng@gmail.com> wrote:
> @@ -754,6 +755,14 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
> break;
> case BTF_KIND_ENUM:
> trace_seq_printf(s, "%ld", arg);
> + for_each_enum(i, t, enump) {
> + if (arg == enump->val) {
> + trace_seq_printf(s, " [%s]",
> + btf_name_by_offset(btf,
> + enump->name_off));
> + break;
> + }
> + }
> break;
I have to ask; how big is that enum list?
Do we really want to do a linear search for every enum we come across?
What we could do is for the first time we hit an enum, create an array of
enums, sort them, and do a binary search from then on. This could be saved
in the tracing code itself (in kernel/trace/trace_btf.c), if BPF doesn't
care about it.
-- Steve
^ permalink raw reply
* Re: [PATCH v2] tracing: Fix funcgraph_exit calltime/rettime offset for 32-bit ARM
From: Steven Rostedt @ 2026-02-02 16:08 UTC (permalink / raw)
To: jempty.liang
Cc: mhiramat, mark.rutland, mathieu.desnoyers, linux-kernel,
linux-trace-kernel
In-Reply-To: <20260202103804.40cbaa27@gandalf.local.home>
On Mon, 2 Feb 2026 10:38:04 -0500
Steven Rostedt <rostedt@goodmis.org> wrote:
> Can you try this patch to see if it fixes the issue for you?
Ignore that patch, try this one instead. This was fixed for trace_events a
while ago, but the same fix wasn't done for ftrace events.
-- Steve
diff --git a/kernel/trace/trace_export.c b/kernel/trace/trace_export.c
index 1698fc22afa0..5b96ac750049 100644
--- a/kernel/trace/trace_export.c
+++ b/kernel/trace/trace_export.c
@@ -14,6 +14,9 @@
#include "trace_output.h"
+/* The alignment of a type when in a structure */
+#define ALIGN_STRUCTFIELD(type) ((int)(__alignof__(struct {type b;})))
+
/* Stub function for events with triggers */
static int ftrace_event_register(struct trace_event_call *call,
enum trace_reg type, void *data)
@@ -88,7 +91,7 @@ static void __always_unused ____ftrace_check_##name(void) \
#undef __field_ext
#define __field_ext(_type, _item, _filter_type) { \
.type = #_type, .name = #_item, \
- .size = sizeof(_type), .align = __alignof__(_type), \
+ .size = sizeof(_type), .align = ALIGN_STRUCTFIELD(_type), \
is_signed_type(_type), .filter_type = _filter_type },
^ permalink raw reply related
* Re: [PATCH bpf-next] ftrace: Fix direct_functions leak in update_ftrace_direct_del
From: patchwork-bot+netdevbpf @ 2026-02-02 16:00 UTC (permalink / raw)
To: Jiri Olsa
Cc: rostedt, ast, alexei.starovoitov, bpf, linux-kernel,
linux-trace-kernel, daniel, andrii, menglong8.dong, song
In-Reply-To: <20260202075849.1684369-1-jolsa@kernel.org>
Hello:
This patch was applied to bpf/bpf-next.git (master)
by Alexei Starovoitov <ast@kernel.org>:
On Mon, 2 Feb 2026 08:58:49 +0100 you wrote:
> Alexei reported memory leak in update_ftrace_direct_del.
> We miss cleanup of the replaced direct_functions in the
> success path in update_ftrace_direct_del, adding that.
>
> Fixes: 8d2c1233f371 ("ftrace: Add update_ftrace_direct_del function")
> Reported-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
> Closes: https://lore.kernel.org/bpf/aX_BxG5EJTJdCMT9@krava/T/#m7c13f5a95f862ed7ab78e905fbb678d635306a0c
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
>
> [...]
Here is the summary with links:
- [bpf-next] ftrace: Fix direct_functions leak in update_ftrace_direct_del
https://git.kernel.org/bpf/bpf-next/c/6b95cc562de2
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply
* Re: [PATCH v2] tracing: Fix funcgraph_exit calltime/rettime offset for 32-bit ARM
From: Steven Rostedt @ 2026-02-02 15:38 UTC (permalink / raw)
To: jempty.liang
Cc: mhiramat, mark.rutland, mathieu.desnoyers, linux-kernel,
linux-trace-kernel
In-Reply-To: <20260202123342.2544795-1-imntjempty@163.com>
On Mon, 2 Feb 2026 12:33:42 +0000
"jempty.liang" <imntjempty@163.com> wrote:
> Commit <66611c0475709607f398e2a5d691b1fc72fe9dfc>
> (fgraph: Remove calltime and rettime from generic)
> incorrectly modified the offset values for calltime and rettime fields
> in the funcgraph_exit traceevent on 32-bit ARM, which are used to parse
> the corresponding values fromtrace rawdata. The actual memory offset of
> calltime is 20 (not 24), and rettime is 28 (not 32) for the
> funcgraph_exit event.
OK, so this is a 32bit issue and not an ARM one. I was able to reproduce it
on 32bit x86 too.
Basically the problem is that the structure used to output the field offset
is out of sync with the actual fields of the structure.
>
> Before the fix,the funcgraph_exit format was:
>
> ~# cat /sys/kernel/tracing/events/ftrace/funcgraph_exit/format
>
> name: funcgraph_exit
> ID: 10
> format:
> ...
> field:unsigned long long calltime; offset:24; size:8; signed:0;
> field:unsigned long long rettime; offset:32; size:8; signed:0;
>
> After the fix, the correct funcgraph_exit format is:
>
> name: funcgraph_exit
> ID: 10
> format:
> ...
> field:unsigned long long calltime; offset:20; size:8; signed:0;
> field:unsigned long long rettime; offset:28; size:8; signed:0;
>
Thus, the way the calltime and rettime are defined is via:
> --- a/kernel/trace/trace_entries.h
> +++ b/kernel/trace/trace_entries.h
> @@ -127,8 +127,8 @@ FTRACE_ENTRY_PACKED(funcgraph_exit, ftrace_graph_ret_entry,
> __field_packed( unsigned long, ret, retval )
> __field_packed( unsigned int, ret, depth )
> __field_packed( unsigned int, ret, overrun )
> - __field(unsigned long long, calltime )
> - __field(unsigned long long, rettime )
The __field() macro.
> + __field_packed(unsigned long long, ret, calltime)
> + __field_packed(unsigned long long, ret, rettime)
You converted it to a __field_packed() macro. The reason this worked is
because fields within a structure defined by __field_packed() has an
alignment of "1" to pack it.
Thus, your "fix" is simply hiding the real bug, which is that the alignment
algorithm is wrong.
Can you try this patch to see if it fixes the issue for you?
Thanks,
-- Steve
diff --git a/kernel/trace/trace_export.c b/kernel/trace/trace_export.c
index 1698fc22afa0..68ef39cf0710 100644
--- a/kernel/trace/trace_export.c
+++ b/kernel/trace/trace_export.c
@@ -88,7 +88,9 @@ static void __always_unused ____ftrace_check_##name(void) \
#undef __field_ext
#define __field_ext(_type, _item, _filter_type) { \
.type = #_type, .name = #_item, \
- .size = sizeof(_type), .align = __alignof__(_type), \
+ .size = sizeof(_type), \
+ .align = __alignof__(_type) > __alignof__(long) ? __alignof__(long) :\
+ __alignof__(_type), \
is_signed_type(_type), .filter_type = _filter_type },
^ permalink raw reply related
* Re: [PATCH v2] tracing/dma: Cap dma_map_sg tracepoint arrays to prevent buffer overflow
From: Marek Szyprowski @ 2026-02-02 15:34 UTC (permalink / raw)
To: Deepanshu Kartikey, rostedt, mhiramat, mathieu.desnoyers,
ptesarik
Cc: jgg, leon, kbusch, sean.anderson, linux-kernel,
linux-trace-kernel, syzbot+28cea38c382fd15e751a
In-Reply-To: <20260130155215.69737-1-kartikey406@gmail.com>
On 30.01.2026 16:52, Deepanshu Kartikey wrote:
> The dma_map_sg tracepoint can trigger a perf buffer overflow when
> tracing large scatter-gather lists. With devices like virtio-gpu
> creating large DRM buffers, nents can exceed 1000 entries, resulting
> in:
>
> phys_addrs: 1000 * 8 bytes = 8,000 bytes
> dma_addrs: 1000 * 8 bytes = 8,000 bytes
> lengths: 1000 * 4 bytes = 4,000 bytes
> Total: ~20,000 bytes
>
> This exceeds PERF_MAX_TRACE_SIZE (8192 bytes), causing:
>
> WARNING: CPU: 0 PID: 5497 at kernel/trace/trace_event_perf.c:405
> perf buffer not large enough, wanted 24620, have 8192
>
> Cap all three dynamic arrays at 128 entries using min() in the array
> size calculation. This ensures arrays are only as large as needed
> (up to the cap), avoiding unnecessary memory allocation for small
> operations while preventing overflow for large ones.
>
> The tracepoint now records the full nents/ents counts and a truncated
> flag so users can see when data has been capped.
>
> Changes in v2:
> - Use min(nents, DMA_TRACE_MAX_ENTRIES) for dynamic array sizing
> instead of fixed DMA_TRACE_MAX_ENTRIES allocation (feedback from
> Steven Rostedt)
> - This allocates only what's needed up to the cap, avoiding waste
> for small operations
>
> Reported-by: syzbot+28cea38c382fd15e751a@syzkaller.appspotmail.com
> Closes: https://protect2.fireeye.com/v1/url?k=6e1b074b-0f901278-6e1a8c04-000babff9bb7-c332a06151070595&q=1&e=71ff52e1-1daa-4a0f-81b1-6593694574b3&u=https%3A%2F%2Fsyzkaller.appspot.com%2Fbug%3Fextid%3D28cea38c382fd15e751a
> Tested-by: syzbot+28cea38c382fd15e751a@syzkaller.appspotmail.com
> Signed-off-by: Deepanshu Kartikey <Kartikey406@gmail.com>
Applied to dma-mapping-fixes, thanks!
> ---
> include/trace/events/dma.h | 25 +++++++++++++++++++------
> 1 file changed, 19 insertions(+), 6 deletions(-)
>
> diff --git a/include/trace/events/dma.h b/include/trace/events/dma.h
> index b3fef140ae15..33e99e792f1a 100644
> --- a/include/trace/events/dma.h
> +++ b/include/trace/events/dma.h
> @@ -275,6 +275,8 @@ TRACE_EVENT(dma_free_sgt,
> sizeof(u64), sizeof(u64)))
> );
>
> +#define DMA_TRACE_MAX_ENTRIES 128
> +
> TRACE_EVENT(dma_map_sg,
> TP_PROTO(struct device *dev, struct scatterlist *sgl, int nents,
> int ents, enum dma_data_direction dir, unsigned long attrs),
> @@ -282,9 +284,12 @@ TRACE_EVENT(dma_map_sg,
>
> TP_STRUCT__entry(
> __string(device, dev_name(dev))
> - __dynamic_array(u64, phys_addrs, nents)
> - __dynamic_array(u64, dma_addrs, ents)
> - __dynamic_array(unsigned int, lengths, ents)
> + __field(int, full_nents)
> + __field(int, full_ents)
> + __field(bool, truncated)
> + __dynamic_array(u64, phys_addrs, min(nents, DMA_TRACE_MAX_ENTRIES))
> + __dynamic_array(u64, dma_addrs, min(ents, DMA_TRACE_MAX_ENTRIES))
> + __dynamic_array(unsigned int, lengths, min(ents, DMA_TRACE_MAX_ENTRIES))
> __field(enum dma_data_direction, dir)
> __field(unsigned long, attrs)
> ),
> @@ -292,11 +297,16 @@ TRACE_EVENT(dma_map_sg,
> TP_fast_assign(
> struct scatterlist *sg;
> int i;
> + int traced_nents = min_t(int, nents, DMA_TRACE_MAX_ENTRIES);
> + int traced_ents = min_t(int, ents, DMA_TRACE_MAX_ENTRIES);
>
> __assign_str(device);
> - for_each_sg(sgl, sg, nents, i)
> + __entry->full_nents = nents;
> + __entry->full_ents = ents;
> + __entry->truncated = (nents > DMA_TRACE_MAX_ENTRIES) || (ents > DMA_TRACE_MAX_ENTRIES);
> + for_each_sg(sgl, sg, traced_nents, i)
> ((u64 *)__get_dynamic_array(phys_addrs))[i] = sg_phys(sg);
> - for_each_sg(sgl, sg, ents, i) {
> + for_each_sg(sgl, sg, traced_ents, i) {
> ((u64 *)__get_dynamic_array(dma_addrs))[i] =
> sg_dma_address(sg);
> ((unsigned int *)__get_dynamic_array(lengths))[i] =
> @@ -306,9 +316,12 @@ TRACE_EVENT(dma_map_sg,
> __entry->attrs = attrs;
> ),
>
> - TP_printk("%s dir=%s dma_addrs=%s sizes=%s phys_addrs=%s attrs=%s",
> + TP_printk("%s dir=%s nents=%d/%d ents=%d/%d%s dma_addrs=%s sizes=%s phys_addrs=%s attrs=%s",
> __get_str(device),
> decode_dma_data_direction(__entry->dir),
> + min_t(int, __entry->full_nents, DMA_TRACE_MAX_ENTRIES), __entry->full_nents,
> + min_t(int, __entry->full_ents, DMA_TRACE_MAX_ENTRIES), __entry->full_ents,
> + __entry->truncated ? " [TRUNCATED]" : "",
> __print_array(__get_dynamic_array(dma_addrs),
> __get_dynamic_array_len(dma_addrs) /
> sizeof(u64), sizeof(u64)),
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
^ permalink raw reply
* Re: [PATCH bpf-next] ftrace: Fix direct_functions leak in update_ftrace_direct_del
From: Steven Rostedt @ 2026-02-02 14:55 UTC (permalink / raw)
To: Jiri Olsa
Cc: Alexei Starovoitov, Alexei Starovoitov, bpf, linux-kernel,
linux-trace-kernel, Daniel Borkmann, Andrii Nakryiko,
Menglong Dong, Song Liu
In-Reply-To: <20260202075849.1684369-1-jolsa@kernel.org>
On Mon, 2 Feb 2026 08:58:49 +0100
Jiri Olsa <jolsa@kernel.org> wrote:
> Alexei reported memory leak in update_ftrace_direct_del.
> We miss cleanup of the replaced direct_functions in the
> success path in update_ftrace_direct_del, adding that.
>
> Fixes: 8d2c1233f371 ("ftrace: Add update_ftrace_direct_del function")
> Reported-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
> Closes: https://lore.kernel.org/bpf/aX_BxG5EJTJdCMT9@krava/T/#m7c13f5a95f862ed7ab78e905fbb678d635306a0c
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
-- Steve
> ---
> kernel/trace/ftrace.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index 8574932e66b6..b12dbd93ae1c 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -6537,6 +6537,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash)
> /* free the new_direct_functions */
> old_direct_functions = new_direct_functions;
> } else {
> + old_direct_functions = direct_functions;
> rcu_assign_pointer(direct_functions, new_direct_functions);
> }
>
^ permalink raw reply
* [PATCH v2] tracing: Fix funcgraph_exit calltime/rettime offset for 32-bit ARM
From: jempty.liang @ 2026-02-02 12:33 UTC (permalink / raw)
To: rostedt, mhiramat, mark.rutland, mathieu.desnoyers
Cc: linux-kernel, linux-trace-kernel, jempty.liang
Commit <66611c0475709607f398e2a5d691b1fc72fe9dfc>
(fgraph: Remove calltime and rettime from generic)
incorrectly modified the offset values for calltime and rettime fields
in the funcgraph_exit traceevent on 32-bit ARM, which are used to parse
the corresponding values fromtrace rawdata. The actual memory offset of
calltime is 20 (not 24), and rettime is 28 (not 32) for the
funcgraph_exit event.
Before the fix,the funcgraph_exit format was:
~# cat /sys/kernel/tracing/events/ftrace/funcgraph_exit/format
name: funcgraph_exit
ID: 10
format:
...
field:unsigned long long calltime; offset:24; size:8; signed:0;
field:unsigned long long rettime; offset:32; size:8; signed:0;
After the fix, the correct funcgraph_exit format is:
name: funcgraph_exit
ID: 10
format:
...
field:unsigned long long calltime; offset:20; size:8; signed:0;
field:unsigned long long rettime; offset:28; size:8; signed:0;
Signed-off-by: jempty.liang <imntjempty@163.com>
---
include/linux/ftrace.h | 2 ++
kernel/trace/trace.h | 3 +--
kernel/trace/trace_entries.h | 8 +++----
kernel/trace/trace_functions_graph.c | 31 +++++++++++++---------------
kernel/trace/trace_irqsoff.c | 5 +++--
kernel/trace/trace_sched_wakeup.c | 6 ++++--
6 files changed, 28 insertions(+), 27 deletions(-)
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index a3a8989e3268..52727a342273 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -1191,6 +1191,8 @@ struct ftrace_graph_ret {
int depth;
/* Number of functions that overran the depth limit for current task */
unsigned int overrun;
+ unsigned long long calltime;
+ unsigned long long rettime;
} __packed;
struct fgraph_ops;
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 69e7defba6c6..18c8a0b1ecd5 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -968,8 +968,7 @@ extern int __trace_graph_retaddr_entry(struct trace_array *tr,
struct ftrace_regs *fregs);
extern void __trace_graph_return(struct trace_array *tr,
struct ftrace_graph_ret *trace,
- unsigned int trace_ctx,
- u64 calltime, u64 rettime);
+ unsigned int trace_ctx);
extern void init_array_fgraph_ops(struct trace_array *tr, struct ftrace_ops *ops);
extern int allocate_fgraph_ops(struct trace_array *tr, struct ftrace_ops *ops);
diff --git a/kernel/trace/trace_entries.h b/kernel/trace/trace_entries.h
index f6a8d29c0d76..362a757e65a2 100644
--- a/kernel/trace/trace_entries.h
+++ b/kernel/trace/trace_entries.h
@@ -127,8 +127,8 @@ FTRACE_ENTRY_PACKED(funcgraph_exit, ftrace_graph_ret_entry,
__field_packed( unsigned long, ret, retval )
__field_packed( unsigned int, ret, depth )
__field_packed( unsigned int, ret, overrun )
- __field(unsigned long long, calltime )
- __field(unsigned long long, rettime )
+ __field_packed(unsigned long long, ret, calltime)
+ __field_packed(unsigned long long, ret, rettime)
),
F_printk("<-- %ps (%u) (start: %llx end: %llx) over: %u retval: %lx",
@@ -149,8 +149,8 @@ FTRACE_ENTRY_PACKED(funcgraph_exit, ftrace_graph_ret_entry,
__field_packed( unsigned long, ret, func )
__field_packed( unsigned int, ret, depth )
__field_packed( unsigned int, ret, overrun )
- __field(unsigned long long, calltime )
- __field(unsigned long long, rettime )
+ __field_packed(unsigned long long, ret, calltime)
+ __field_packed(unsigned long long, ret, rettime)
),
F_printk("<-- %ps (%u) (start: %llx end: %llx) over: %u",
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index 1de6f1573621..0d2266ec67a4 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -317,10 +317,12 @@ __trace_graph_function(struct trace_array *tr,
struct ftrace_graph_ret ret = {
.func = ip,
.depth = 0,
+ .calltime = time,
+ .rettime = time,
};
__trace_graph_entry(tr, &ent, trace_ctx);
- __trace_graph_return(tr, &ret, trace_ctx, time, time);
+ __trace_graph_return(tr, &ret, trace_ctx);
}
void
@@ -333,8 +335,7 @@ trace_graph_function(struct trace_array *tr,
void __trace_graph_return(struct trace_array *tr,
struct ftrace_graph_ret *trace,
- unsigned int trace_ctx,
- u64 calltime, u64 rettime)
+ unsigned int trace_ctx)
{
struct ring_buffer_event *event;
struct trace_buffer *buffer = tr->array_buffer.buffer;
@@ -346,8 +347,6 @@ void __trace_graph_return(struct trace_array *tr,
return;
entry = ring_buffer_event_data(event);
entry->ret = *trace;
- entry->calltime = calltime;
- entry->rettime = rettime;
trace_buffer_unlock_commit_nostack(buffer, event);
}
@@ -372,10 +371,9 @@ void trace_graph_return(struct ftrace_graph_ret *trace,
struct trace_array *tr = gops->private;
struct fgraph_times *ftimes;
unsigned int trace_ctx;
- u64 calltime, rettime;
int size;
- rettime = trace_clock_local();
+ trace->rettime = trace_clock_local();
ftrace_graph_addr_finish(gops, trace);
@@ -390,10 +388,10 @@ void trace_graph_return(struct ftrace_graph_ret *trace,
handle_nosleeptime(tr, trace, ftimes, size);
- calltime = ftimes->calltime;
+ trace->calltime = ftimes->calltime;
trace_ctx = tracing_gen_ctx();
- __trace_graph_return(tr, trace, trace_ctx, calltime, rettime);
+ __trace_graph_return(tr, trace, trace_ctx);
}
static void trace_graph_thresh_return(struct ftrace_graph_ret *trace,
@@ -418,8 +416,10 @@ static void trace_graph_thresh_return(struct ftrace_graph_ret *trace,
tr = gops->private;
handle_nosleeptime(tr, trace, ftimes, size);
+ trace->calltime = ftimes->calltime;
+
if (tracing_thresh &&
- (trace_clock_local() - ftimes->calltime < tracing_thresh))
+ (trace->rettime - ftimes->calltime < tracing_thresh))
return;
else
trace_graph_return(trace, gops, fregs);
@@ -956,7 +956,7 @@ print_graph_entry_leaf(struct trace_iterator *iter,
graph_ret = &ret_entry->ret;
call = &entry->graph_ent;
- duration = ret_entry->rettime - ret_entry->calltime;
+ duration = graph_ret->rettime - graph_ret->calltime;
if (data) {
struct fgraph_cpu_data *cpu_data;
@@ -1275,14 +1275,11 @@ print_graph_entry(struct ftrace_graph_ent_entry *field, struct trace_seq *s,
}
static enum print_line_t
-print_graph_return(struct ftrace_graph_ret_entry *retentry, struct trace_seq *s,
+print_graph_return(struct ftrace_graph_ret *trace, struct trace_seq *s,
struct trace_entry *ent, struct trace_iterator *iter,
u32 flags)
{
- struct ftrace_graph_ret *trace = &retentry->ret;
- u64 calltime = retentry->calltime;
- u64 rettime = retentry->rettime;
- unsigned long long duration = rettime - calltime;
+ unsigned long long duration = trace->rettime - trace->calltime;
struct fgraph_data *data = iter->private;
struct trace_array *tr = iter->tr;
unsigned long func;
@@ -1482,7 +1479,7 @@ print_graph_function_flags(struct trace_iterator *iter, u32 flags)
case TRACE_GRAPH_RET: {
struct ftrace_graph_ret_entry *field;
trace_assign_type(field, entry);
- return print_graph_return(field, s, entry, iter, flags);
+ return print_graph_return(&field->ret, s, entry, iter, flags);
}
case TRACE_STACK:
case TRACE_FN:
diff --git a/kernel/trace/trace_irqsoff.c b/kernel/trace/trace_irqsoff.c
index 17673905907c..946be462a211 100644
--- a/kernel/trace/trace_irqsoff.c
+++ b/kernel/trace/trace_irqsoff.c
@@ -229,11 +229,12 @@ static void irqsoff_graph_return(struct ftrace_graph_ret *trace,
if (!func_prolog_dec(tr, &data, &flags))
return;
- rettime = trace_clock_local();
+ trace->rettime = trace_clock_local();
calltime = fgraph_retrieve_data(gops->idx, &size);
if (calltime) {
+ trace->calltime = *calltime;
trace_ctx = tracing_gen_ctx_flags(flags);
- __trace_graph_return(tr, trace, trace_ctx, *calltime, rettime);
+ __trace_graph_return(tr, trace, trace_ctx);
}
local_dec(&data->disabled);
}
diff --git a/kernel/trace/trace_sched_wakeup.c b/kernel/trace/trace_sched_wakeup.c
index 8faa73d3bba1..3bcfd1bf60ad 100644
--- a/kernel/trace/trace_sched_wakeup.c
+++ b/kernel/trace/trace_sched_wakeup.c
@@ -164,11 +164,13 @@ static void wakeup_graph_return(struct ftrace_graph_ret *trace,
if (!func_prolog_preempt_disable(tr, &data, &trace_ctx))
return;
- rettime = trace_clock_local();
+ trace->rettime = trace_clock_local();
calltime = fgraph_retrieve_data(gops->idx, &size);
- if (calltime)
+ if (calltime) {
+ trace->calltime = *calltime;
__trace_graph_return(tr, trace, trace_ctx, *calltime, rettime);
+ }
local_dec(&data->disabled);
preempt_enable_notrace();
--
2.25.1
^ permalink raw reply related
* Re: [PATCH] tracing: remove __printf() attribute on __ftrace_vbprintk()
From: Jeff Layton @ 2026-02-02 11:37 UTC (permalink / raw)
To: Arnd Bergmann, Steven Rostedt, Masami Hiramatsu, Simon Horman,
Anna Schumaker, Chuck Lever
Cc: Arnd Bergmann, Mathieu Desnoyers, Andrew Morton, Andy Shevchenko,
Randy Dunlap, Yury Norov, Joel Fernandes, linux-kernel,
linux-trace-kernel
In-Reply-To: <20260202095834.1328352-1-arnd@kernel.org>
On Mon, 2026-02-02 at 10:58 +0100, Arnd Bergmann wrote:
> From: Arnd Bergmann <arnd@arndb.de>
>
> The sunrpc change to use trace_printk() for debugging caused
> a new warning for every instance of dprintk() in some configurations,
> when -Wformat-security is enabled:
>
> fs/nfs/getroot.c: In function 'nfs_get_root':
> fs/nfs/getroot.c:90:17: error: format not a string literal and no format arguments [-Werror=format-security]
> 90 | nfs_errorf(fc, "NFS: Couldn't getattr on root");
>
> I've been slowly chipping away at those warnings over time with the
> intention of enabling them by default in the future. While I could not
> figure out why this only happens for this one instance, I see that the
> __trace_bprintk() function is always called with a local variable as
> the format string, rather than a literal.
>
> Remove the __printf(2,3) annotation on this function, as this is can
> only be validated for literals. The format strings still get checked
> because the underlying literal keeps getting passed into __trace_printk()
> in the "else" branch, which is not taken but still evaluated for
> compile-time warnings.
>
> Fixes: ec7d8e68ef0e ("sunrpc: add a Kconfig option to redirect dfprintk() output to trace buffer")
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
> include/linux/trace_printk.h | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/include/linux/trace_printk.h b/include/linux/trace_printk.h
> index bb5874097f24..2670ec7f4262 100644
> --- a/include/linux/trace_printk.h
> +++ b/include/linux/trace_printk.h
> @@ -107,7 +107,6 @@ do { \
> __trace_printk(_THIS_IP_, fmt, ##args); \
> } while (0)
>
> -extern __printf(2, 3)
> int __trace_bprintk(unsigned long ip, const char *fmt, ...);
>
> extern __printf(2, 3)
Acked-by: Jeff Layton <jlayton@kernel.org>
^ permalink raw reply
* [PATCH 2/2] tracing: resolve enum names for function arguments via BTF
From: Donglin Peng @ 2026-02-02 11:15 UTC (permalink / raw)
To: rostedt, ast
Cc: mhiramat, andrii.nakryiko, linux-kernel, Donglin Peng,
linux-trace-kernel
In-Reply-To: <20260202111548.3555306-1-dolinux.peng@gmail.com>
From: Donglin Peng <pengdonglin@xiaomi.com>
Use BTF to print symbolic names for enum-type function arguments,
improving trace readability.
Before:
count_memcg_events(memcg=0xffff..., idx=20, count=0x1) {
After:
count_memcg_events(memcg=0xffff..., idx=20 [PGFAULT], count=0x1) {
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: linux-trace-kernel@vger.kernel.org
Signed-off-by: Donglin Peng <pengdonglin@xiaomi.com>
---
kernel/trace/trace_output.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_output.c b/kernel/trace/trace_output.c
index cc2d3306bb60..c395f768c3b8 100644
--- a/kernel/trace/trace_output.c
+++ b/kernel/trace/trace_output.c
@@ -695,12 +695,13 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
{
const struct btf_param *param;
const struct btf_type *t;
+ const struct btf_enum *enump;
const char *param_name;
char name[KSYM_NAME_LEN];
unsigned long arg;
struct btf *btf;
s32 tid, nr = 0;
- int a, p, x;
+ int a, p, x, i;
u16 encode;
trace_seq_printf(s, "(");
@@ -754,6 +755,14 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
break;
case BTF_KIND_ENUM:
trace_seq_printf(s, "%ld", arg);
+ for_each_enum(i, t, enump) {
+ if (arg == enump->val) {
+ trace_seq_printf(s, " [%s]",
+ btf_name_by_offset(btf,
+ enump->name_off));
+ break;
+ }
+ }
break;
default:
/* This does not handle complex arguments */
--
2.34.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox