From: Pratyush Yadav <pratyush@kernel.org>
To: Tarun Sahu <tarunsahu@google.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>,
Mike Rapoport <rppt@kernel.org>,
robin.murphy@arm.com, maz@kernel.org, will@kernel.org,
David Matlack <dmatlack@google.com>,
Alexander Graf <graf@amazon.com>,
seanjc@google.com, jgg@nvidia.com, loganodell@google.com,
arnd@arndb.de, dwmw2@infradead.org, baolu.lu@linux.intel.com,
Bjorn Helgaas <bhelgaas@google.com>,
oupton@kernel.org, alex@shazbot.org, gle.com@google.com,
pbonzini@redhat.com, Andrew Morton <akpm@linux-foundation.org>,
kevin.tian@intel.com, joro@8bytes.org, kvmarm@lists.linux.dev,
linux-pci@vger.kernel.org, Pratyush Yadav <pratyush@kernel.org>,
kexec@lists.infradead.org, kvm@vger.kernel.org,
iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org,
skhawaja@google.com, sagis@google.com,
linux-arch@vger.kernel.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v4 03/19] kho, memfd_luo: Reference struct kho_vmalloc via serialized pointer
Date: Fri, 20 Nov 2026 18:19:35 +0000 [thread overview]
Message-ID: <2vxz4idb1i2w.fsf@kernel.org> (raw)
In-Reply-To: <20261110050202.529473-4-tarunsahu@google.com> (Tarun Sahu's message of "Tue, 10 Nov 2026 05:01:46 +0000")
On Tue, Nov 10 2026, Tarun Sahu wrote:
> Subsystems preserving vmalloc allocations currently embed struct
> kho_vmalloc directly by value in their serialized structures or device
> tree properties. For example, struct memfd_luo_ser embeds struct
> kho_vmalloc folios, and test_kho writes the structure directly into the
> folios_info FDT property.
>
> Embedding struct kho_vmalloc directly couples the containing layout's
> field offsets and size to struct kho_vmalloc. Any modification to struct
> kho_vmalloc shifts subsequent fields in the containing structure and
> breaks the handover ABI across kexec.
>
> Convert callers to hold a DECLARE_KHOSER_PTR pointer to struct
> kho_vmalloc instead of embedding it by value. In memfd_luo, allocate the
> main structure and struct kho_vmalloc in the same preserved folio, placing
> struct kho_vmalloc in the remaining 8-byte-aligned space, and bump the
> compatibility string to memfd-v3. In test_kho, place struct kho_vmalloc
> in the remaining space of the FDT folio, store the serialized pointer in
> the folios_info property, and bump the compatibility string to
> kho-test-v2.
>
> Signed-off-by: Tarun Sahu <tarunsahu@google.com>
> ---
> include/linux/kho/abi/kexec_handover.h | 6 +++
> include/linux/kho/abi/memfd.h | 8 ++--
> lib/test_kho.c | 29 ++++++++----
> mm/memfd_luo.c | 63 ++++++++++++++++----------
> 4 files changed, 68 insertions(+), 38 deletions(-)
>
[...]
> diff --git a/mm/memfd_luo.c b/mm/memfd_luo.c
> index 6b68220275eb..4471cced57db 100644
> --- a/mm/memfd_luo.c
> +++ b/mm/memfd_luo.c
> @@ -85,9 +85,8 @@
> #include "internal.h"
>
> static int memfd_luo_preserve_folios(struct file *file,
> - struct kho_vmalloc *kho_vmalloc,
> - struct memfd_luo_folio_ser **out_folios_ser,
> - u64 *nr_foliosp)
> + struct memfd_luo_ser *ser,
> + struct memfd_luo_folio_ser **out_folios_ser)
> {
> struct inode *inode = file_inode(file);
> struct memfd_luo_folio_ser *folios_ser;
> @@ -104,7 +103,8 @@ static int memfd_luo_preserve_folios(struct file *file,
> * are not set.
> */
> if (!size) {
> - *nr_foliosp = 0;
> + ser->folios.phys = 0;
I don't think it is a good idea to access this directly.
Please take this patch instead:
https://lore.kernel.org/kexec/20261116134013.1636689-2-pratyush@kernel.org/
> + ser->nr_folios = 0;
> *out_folios_ser = NULL;
> return 0;
> }
> @@ -200,12 +200,12 @@ static int memfd_luo_preserve_folios(struct file *file,
> pfolio->index = folio->index;
> }
>
> - err = kho_preserve_vmalloc(folios_ser, kho_vmalloc);
> + err = kho_preserve_vmalloc(folios_ser, KHOSER_LOAD_PTR(ser->folios));
> if (err)
> goto err_unpreserve;
>
> kvfree(folios);
> - *nr_foliosp = nr_folios;
> + ser->nr_folios = nr_folios;
> *out_folios_ser = folios_ser;
>
> /*
> @@ -254,37 +254,43 @@ static void memfd_luo_unpreserve_folios(struct kho_vmalloc *kho_vmalloc,
> vfree(folios_ser);
> }
>
> +static size_t memfd_luo_ser_size(void)
> +{
> + return sizeof(struct memfd_luo_ser) + sizeof(struct kho_vmalloc);
> +}
> +
> +static struct memfd_luo_ser *memfd_luo_ser_init(void *mem)
> +{
> + struct kho_vmalloc *kho_vmalloc = mem + sizeof(struct memfd_luo_ser);
> + struct memfd_luo_ser *ser = mem;
> +
> + KHOSER_STORE_PTR(ser->folios, kho_vmalloc);
> + return ser;
> +}
> +
Hmm, I am not a huge fan of this. This makes doing such allocations
quite annoying, and I reckon we will be seeing a fair bit of them in
many places.
The key observation here is that most file handlers need to allocate a
bunch of preserved memory, but they usually can free it all at once. So
we can get away with a very simple allocator that hands out memory
linearly.
I have sketched out some code for this, take a look at the patch below.
Note that it is only compile tested.
Perhaps give it a try and see if it does the job better.
> static int memfd_luo_preserve(struct liveupdate_file_op_args *args)
> {
> struct inode *inode = file_inode(args->file);
> struct memfd_luo_folio_ser *folios_ser;
> struct memfd_luo_ser *ser;
> - u64 nr_folios, inode_size;
> int err = 0, seals;
> + u64 inode_size;
>
> inode_lock(inode);
> shmem_freeze(inode, true);
>
> - /* Allocate the main serialization structure in preserved memory */
> - ser = kho_alloc_preserve(sizeof(*ser));
> - if (IS_ERR(ser)) {
> - err = PTR_ERR(ser);
> - goto err_unlock;
> - }
> -
> seals = memfd_get_seals(args->file);
> if (seals < 0) {
> err = seals;
> - goto err_free_ser;
> + goto err_unlock;
> }
>
> /* Make sure the file only has the seals supported by this version. */
> if (seals & ~MEMFD_LUO_ALL_SEALS) {
> err = -EOPNOTSUPP;
> - goto err_free_ser;
> + goto err_unlock;
> }
>
> - ser->pos = args->file->f_pos;
> inode_size = i_size_read(inode);
>
> /*
> @@ -293,18 +299,26 @@ static int memfd_luo_preserve(struct liveupdate_file_op_args *args)
> */
> if (DIV_ROUND_UP_ULL(inode_size, PAGE_SIZE) > UINT_MAX) {
> err = -EFBIG;
> - goto err_free_ser;
> + goto err_unlock;
> + }
> +
> + /* Allocate the main serialization structure in preserved memory */
> + ser = kho_alloc_preserve(memfd_luo_ser_size());
> + if (IS_ERR(ser)) {
> + err = PTR_ERR(ser);
> + goto err_unlock;
> }
> + ser = memfd_luo_ser_init(ser);
>
> + ser->pos = args->file->f_pos;
> ser->size = inode_size;
> ser->seals = seals;
> + ser->flags = 0;
Nit: Looks like a stray change. Flags should be initialized to 0 anyway,
right?
>
> - err = memfd_luo_preserve_folios(args->file, &ser->folios,
> - &folios_ser, &nr_folios);
> + err = memfd_luo_preserve_folios(args->file, ser, &folios_ser);
> if (err)
> goto err_free_ser;
>
> - ser->nr_folios = nr_folios;
> inode_unlock(inode);
>
> args->private_data = folios_ser;
[...]
--- 8< ---
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h
index ef2b188c8ed0..f17b46a4dbf6 100644
--- a/include/linux/kexec_handover.h
+++ b/include/linux/kexec_handover.h
@@ -13,6 +13,21 @@ struct kho_vmalloc;
struct folio;
struct page;
+/**
+ * struct kho_arena - Allocator handing out pieces of one preserved buffer.
+ * @buf: Start of the preserved buffer.
+ * @size: Size of the buffer in bytes.
+ * @used: Offset in the buffer where the last allocation ended.
+ *
+ * This is runtime state only. It is not preserved and the next kernel does not
+ * need it to free the buffer.
+ */
+struct kho_arena {
+ void *buf;
+ size_t size;
+ size_t used;
+};
+
#ifdef CONFIG_KEXEC_HANDOVER
bool kho_is_enabled(void);
bool is_kho_boot(void);
@@ -26,6 +41,9 @@ void kho_unpreserve_vmalloc(struct kho_vmalloc *preservation);
void *kho_alloc_preserve(size_t size);
void kho_unpreserve_free(void *mem);
void kho_restore_free(void *mem);
+int kho_arena_init(struct kho_arena *arena, size_t size);
+void *kho_arena_alloc(struct kho_arena *arena, size_t size);
+void kho_arena_destroy(struct kho_arena *arena);
struct folio *kho_restore_folio(phys_addr_t phys);
struct page *kho_restore_pages(phys_addr_t phys, unsigned long nr_pages);
void *kho_restore_vmalloc(const struct kho_vmalloc *preservation);
@@ -89,6 +107,18 @@ static inline void *kho_alloc_preserve(size_t size)
static inline void kho_unpreserve_free(void *mem) { }
static inline void kho_restore_free(void *mem) { }
+static inline int kho_arena_init(struct kho_arena *arena, size_t size)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void *kho_arena_alloc(struct kho_arena *arena, size_t size)
+{
+ return NULL;
+}
+
+static inline void kho_arena_destroy(struct kho_arena *arena) { }
+
static inline struct folio *kho_restore_folio(phys_addr_t phys)
{
return NULL;
diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
index 38e65d210cb9..c2788027d717 100644
--- a/kernel/liveupdate/kexec_handover.c
+++ b/kernel/liveupdate/kexec_handover.c
@@ -1659,6 +1659,78 @@ void kho_restore_free(void *mem)
}
EXPORT_SYMBOL_GPL(kho_restore_free);
+/* Enough for any serialized data structure. */
+#define KHO_ARENA_ALIGN sizeof(u64)
+
+/**
+ * kho_arena_init - Initialize a preserved buffer backing an arena.
+ * @arena: The arena to initialize.
+ * @size: The size of buffer to allocate.
+ *
+ * kho_alloc_preserve() rounds every allocation up to use a full folio. A caller
+ * that needs many small objects wastes most of a folio on each. An arena takes
+ * one buffer from kho_alloc_preserve() and hands out pieces of it one after
+ * another.
+ *
+ * Allocations can not be freed one by one. The whole buffer is freed at once,
+ * with kho_arena_destroy(). or with kho_restore_free() in the incoming kernel.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+int kho_arena_init(struct kho_arena *arena, size_t size)
+{
+ void *buf;
+
+ buf = kho_alloc_preserve(size);
+ if (IS_ERR(buf))
+ return PTR_ERR(buf);
+
+ arena->buf = buf;
+ arena->size = PAGE_SIZE << get_order(size);
+ arena->used = 0;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(kho_arena_init);
+
+/**
+ * kho_arena_alloc - Allocate preserved memory from an arena.
+ * @arena: The arena to allocate from.
+ * @size: The number of bytes to allocate.
+ *
+ * Hands out the next @size bytes of the arena's buffer. The memory is zeroed,
+ * aligned to 8 bytes, and already preserved.
+ *
+ * Return: A pointer to the allocated memory, or NULL if @size is 0 or the
+ * arena does not have @size bytes left.
+ */
+void *kho_arena_alloc(struct kho_arena *arena, size_t size)
+{
+ size_t start = ALIGN(arena->used, KHO_ARENA_ALIGN);
+
+ if (!size || size > arena->size - start)
+ return NULL;
+
+ arena->used = start + size;
+
+ return arena->buf + start;
+}
+EXPORT_SYMBOL_GPL(kho_arena_alloc);
+
+/**
+ * kho_arena_destroy - Unpreserve and free the buffer backing an arena.
+ * @arena: The arena to destroy.
+ *
+ * Frees all the memory allocated from the arena. Must only be called in the
+ * outgoing kernel.
+ */
+void kho_arena_destroy(struct kho_arena *arena)
+{
+ kho_unpreserve_free(arena->buf);
+ *arena = (struct kho_arena){};
+}
+EXPORT_SYMBOL_GPL(kho_arena_destroy);
+
/**
* is_kho_boot - check if current kernel was booted via KHO-enabled
* kexec
--
Regards,
Pratyush Yadav
next prev parent reply other threads:[~2026-11-20 18:19 UTC|newest]
Thread overview: 69+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-10 5:01 [PATCH v4 00/19] kho, liveupdate: Add feature header for fine-grained ABI compatibility Tarun Sahu
2026-11-10 5:01 ` [PATCH v4 01/19] liveupdate, kho: Remove __packed from KHO, LUO and memfd ABI structures Tarun Sahu
2026-11-10 12:56 ` Pasha Tatashin
2026-11-10 17:28 ` tarunsahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 02/19] kho: block: Record the entry size in the block header Tarun Sahu
2026-11-10 13:10 ` Pasha Tatashin
2026-11-11 5:45 ` tarunsahu
2026-11-15 17:35 ` Mike Rapoport
2026-11-10 17:03 ` David Matlack
2026-11-15 6:37 ` tarunsahu
2026-11-11 6:16 ` sashiko-bot
2026-11-15 17:39 ` Mike Rapoport
2026-11-20 16:11 ` Pratyush Yadav
2026-11-10 5:01 ` [PATCH v4 03/19] kho, memfd_luo: Reference struct kho_vmalloc via serialized pointer Tarun Sahu
2026-11-10 14:23 ` Pasha Tatashin
2026-11-11 6:20 ` tarunsahu
2026-11-11 6:16 ` sashiko-bot
2026-11-20 18:19 ` Pratyush Yadav [this message]
2026-11-10 5:01 ` [PATCH v4 04/19] liveupdate: Reference struct luo_file_set_ser " Tarun Sahu
2026-11-10 14:47 ` Pasha Tatashin
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 05/19] kho: Add feature header for fine-grained ABI compatibility Tarun Sahu
2026-11-10 14:47 ` Pasha Tatashin
2026-11-10 18:13 ` tarunsahu
2026-11-10 15:12 ` Pasha Tatashin
2026-11-10 18:25 ` tarunsahu
2026-11-16 4:34 ` Chris Down
2026-11-20 4:00 ` tarunsahu
2026-11-11 6:16 ` sashiko-bot
2026-11-15 18:01 ` Mike Rapoport
2026-11-10 5:01 ` [PATCH v4 06/19] kho: kexec_metadata: Add the feature header Tarun Sahu
2026-11-10 15:08 ` Pasha Tatashin
2026-11-11 6:33 ` tarunsahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 07/19] kho: vmalloc: " Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 08/19] kho: block: Add the feature header to blocks Tarun Sahu
2026-11-10 17:07 ` David Matlack
2026-11-14 9:34 ` tarunsahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 09/19] liveupdate: Add feature header to struct luo_ser Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 10/19] liveupdate: Add feature header to struct luo_session_ser Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 11/19] liveupdate: Add feature header to struct luo_file_set_ser Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 12/19] liveupdate: Add feature header to struct luo_file_ser Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 13/19] liveupdate: Add feature header to struct luo_flb_header_ser Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 14/19] liveupdate: Add feature header to struct luo_flb_ser Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-16 4:34 ` Chris Down
2026-11-10 5:01 ` [PATCH v4 15/19] PCI: liveupdate: Add feature header to struct pci_ser Tarun Sahu
2026-11-10 16:39 ` David Matlack
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:01 ` [PATCH v4 16/19] PCI: liveupdate: Add feature header to struct pci_dev_ser Tarun Sahu
2026-11-10 16:23 ` David Matlack
2026-11-14 17:35 ` tarunsahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:02 ` [PATCH v4 17/19] mm: memfd_luo: Add feature header to struct memfd_luo_ser Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:02 ` [PATCH v4 18/19] mm: memfd_luo: Add feature header to struct memfd_luo_folio_ser Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 5:02 ` [PATCH v4 19/19] kho: Add KUnit test for the feature header helpers Tarun Sahu
2026-11-11 6:16 ` sashiko-bot
2026-11-10 12:48 ` [PATCH v4 00/19] kho, liveupdate: Add feature header for fine-grained ABI compatibility Pasha Tatashin
2026-11-11 5:36 ` tarunsahu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2vxz4idb1i2w.fsf@kernel.org \
--to=pratyush@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=alex@shazbot.org \
--cc=arnd@arndb.de \
--cc=baolu.lu@linux.intel.com \
--cc=bhelgaas@google.com \
--cc=dmatlack@google.com \
--cc=dwmw2@infradead.org \
--cc=gle.com@google.com \
--cc=graf@amazon.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kexec@lists.infradead.org \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.linux.dev \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-pci@vger.kernel.org \
--cc=loganodell@google.com \
--cc=maz@kernel.org \
--cc=oupton@kernel.org \
--cc=pasha.tatashin@soleen.com \
--cc=pbonzini@redhat.com \
--cc=robin.murphy@arm.com \
--cc=rppt@kernel.org \
--cc=sagis@google.com \
--cc=seanjc@google.com \
--cc=skhawaja@google.com \
--cc=tarunsahu@google.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.