Re: [RFC v1 0/9] kho: granular compatibility and header decoupling

[Pasha Tatashin <pasha.tatashin@soleen.com>]

On 06-07 14:58, Mike Rapoport wrote:
> On Fri, 05 Jun 2026 03:32:26 +0000, Pasha Tatashin <pasha.tatashin@soleen.com> wrote:
> 
> Hi,
> 
> > [...]
> > data structure. Keeping all of this within the same `kexec_handover.c`
> > file, and also under the same global version, is no longer sustainable.
> > 
> > To address this, this series:
> > 1. Refactors and reorganizes the code by splitting out radix tree
> >    and vmalloc into separate files.
> 
> I'd keep vmalloc where it is, it's more of a memory preservation primitive
> rather than a data structure of it's own. The data structure it uses is an
> implementation detail.

kho vmalloc is absolutely a data structure. KHO core only provides the
basic handover mechanism (FDT nodes, physical memory ranges). vmalloc
is a structured representation on top of KHO, and should provide its own
versioned ABI.

If we change any of the vmalloc serialized structures (like kho_vmalloc,
kho_vmalloc_chunk, or kho_vmalloc_hdr), then vmalloc won't work and
compatibility will break.

Core KHO does not need vmalloc; nothing in kexec_handover.c uses it.

Instead, vmalloc has external customers:
- memfd (uses it to preserve serialized folio metadata)
- KHO test suite in lib/test_kho.c (uses it to preserve physical address arrays)

> Let's minimize the churn where possible for the sake of git blame and
> backports.

It is much better to do the right cleanups now while KHO is young. Once more
subsystems are added, this refactoring will be twice as hard. Modularizing the
code now guarantees a simpler, safer, and scalable design. Placing each data
structure in its own file gives us code that is easier to maintain, review, and
less prone to bugs.

> > 2. Moves and organizes internal and ABI headers into structured
> >    directories under include/linux/kho/ and include/linux/kho/abi/.
> >    Instead of cluttering include/linux/ with prefix-styled headers like
> >    kho_block.h or kho_radix_tree.h, we use the already existing
> >    include/linux/kho/ directory (e.g., kho/block.h and
> >    kho/radix_tree.h).
> 
> This looks to me like unnecessary churn.
> These all are bundled with KHO anyway, there is no header dependencies
> that justify small headers for each two functions and netiher
> linux/kexec_handover.h nor linux/kho/abi/kexec_handover.h are that long
> to start splitting them.

External users only need to include the headers they actually use. For
example, LUO shouldn't have to pull vmalloc or radix tree KHO
declarations, and memfd does not need block.

From a maintenance point of view, it is much easier to catch ABI
changes when the file with the appropriate version has been changed,
and most likely the version of that file should be updated. If a single
header contains compatibility versions for several different data
structures, it is easier to miss the correct version update.

Since we are splitting the source files (like kho_radix.c and
kho_vmalloc.c), the headers should logically follow the same
modularity.

> 
> > 3. Introduces a standard set of compatibility helpers in
> >    kho/abi/compat.h.
> > 4. Decouples the compatibility strings of individual KHO subsystems
> >    (radix tree, vmalloc, and block) from the global KHO version.
> >    This enables independent, granular compatibility versioning.
> 
> I agree that we should decouple versioning of these components from the
> global KHO versioning.
> Can't say I agree with the way you propose to do it.
> 
> I don't like that each user of a KHO component should include that
> component version in its own version string (or whatever it may become
> later).
> 
> It requires ABI headers update each time a user decides to add a new
> data structure and worse when there is a change to that data structure.
> It creates coupling of the data structure user with its particular
> version and just looks ugly IMHO.

It is actually the opposite.

If a user adds a new data structure, that new data structure will have
its own compatibility version. Instead of the current approach where
the global version string needs to be updated, only the new version
string would be added.

Also, if someone updates their code to use the new data structure, their
compatibility string is going to be updated anyway, as part of using
the data structure requires including the dependency in their
compatibility.


> Suppose we added new fields to vmalloc, but made the implementation of
> restore to be able to cope with both old and new versions. 
> How this would be reflected in memfd versioning?
> We'll add both versions of vmalloc to memfd version? And all other vmalloc
> users?

Backward compatibility is not in scope at the moment, but we can make
the version parsing more granular in the future.

Instead of a simple strncmp(), we can introduce a standard callback
interface for data structures. Each data structure implementation would
implement this interface, and we would pass the parsed version string
to the data-structure-specific version check.

> Or, say, we add support to kmalloc() and use it in kho_block.
> Then we'd have to add kmalloc() versioning to all kho_block users, right?

I was thinking about this. Since we don't have examples of data
structures depending on each other right now, I simply made sure
there are no duplicates in the compatibility strings.

If data structures have interdependencies in the future, we can easily
remove this uniqueness restriction. The users of block will still
include the block compatibility string (which automatically includes
kmalloc), and if user also depends on kmalloc, they will include it
as well.

> I think the versioning of each component should be handled by ->restore()
> of that component. If it sees an incompatible version in the preserved
> data, it returns an error. The versions can be stored e.g. in the base KHO
> fdt.

Hm, I think, checking compatibility inside ->restore() of each component may be
too late in the boot sequence.

By checking the composite compatibility strings upfront (before invoking
the actual restore/retrieve callbacks), we can guarantee that the entire
state configuration is fully compatible. If any mismatch is found, we
can cleanly abort the live update.

Additionally, keeping the versioning managed via composite strings on the
serialized data and registered handlers keeps the KHO core completely
decoupled from individual component ABIs, avoiding the need to bloat the
base KHO FDT with subsystem-specific versions.

> > 5. Adds a KUnit test suite to verify that the composite compatibility
> >    strings of different subsystems remain unique and sorted in
> >    alphabetical order, guaranteeing a consistent and predictable
> >    representation across configurations.
> 
> Without "composite compatibility strings" we don't need to care about
> them "remaining unique and sorted in alphabetical order".

These are not strict runtime requirements; they are simply there to enforce
code cleanliness and prevent human errors like accidental duplicates or
mismatched orders.

Even with a simple strncmp(), it works perfectly fine as long as the
strings match exactly. If the uniqueness or sorting constraints are too
strict, they can easily be removed.

In the future, we can transition to a more sophisticated version checker
that parses the composite string into individual subsystem version tokens
and verifies them one-by-one, rather than relying on a strict literal
strcmp() string comparison.

> The need for this test alone is already a red flag ;-)

I will remove test ;-)