Re: [PATCH v4 05/11] KVM: LUO: Support VM preservation across live updates

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Sep 10, 2026 at 09:03:55AM -0700, Sean Christopherson wrote:

> From that perspective, what I am proposing actually goes a step further.  I'm
> saying don't commit to supporting *any* specific versions in upstream.  Express
> the feature requirements in the serialization payloads, and let userspace sort
> out what kernels are compatible based on their actual usage.  The kernel may need
> to provide additional discovery mechanisms, e.g. so that userspace can probe to
> see what is supported, but discovery is usually fairly simple to implement and
> maintain.

I'm not so fussed about the version number scheme itself. Alot of
different schemes have been proposed over the last year and half, and
some were very complicated. Nobody came with a more granular proposal
that also didn't have alot of complexity attached to it.

FWWI this started out with creating device tree fragments with a full
YAML schema, and the same kind of perfect ABI like you are talking
about. Yet it did not figure out how to make it discoverable, it was
super inefficient and very hard to code for.

It has to be discoverable from the ELF, restrictable on the export
side, easy on maintainers, and not complex to implement.

> No, the subsystem just needs to make sure that it serializes its data using the
> defined ABI (where ABI here means the format of the payload and the meaning of
> any flags in the header).  That should be *easier* for maintainers to handle than
> trying to support arbitrary versions, because it eliminates subjectivity and
> having to make judgment calls or remember magic version numbers.

I'm pretty sure I have something like PTSD from ABI definitions
adventures on the uAPI side. :( Please don't call it easier!

I really don't want more of those in the kernel process. I think
Linus's non-stable-api-nonsense is really a good thing for community
health.

With the simplification that upstream supports only one version at
once, a simple "id" to represent that ABI was the simplest, easiest on
maintainers thing. There is never a fight. Someone has a new idea,
great no worries, change the ID. Done.

I guess I should say my perspective is to prioritize not burdening the
maintainers.

> > I was told KVM had the smallest luo footprint of everything, so
> > perhaps your perspective is different.
> 
> Only because KVM already has a massive ABI surface for save/restore.  

Yeah, you are lucky, other subsystems haven't done that. Honestly, was
it easy to create?

> If you want to convince me that magic version numbers are the right
> approach, then show me how KVM's existing save/restore support would
> be made "better" and easier to maintain by throwing away all of
> KVM's save/restore uAPI and replacing it with a versioning scheme.

I don't know about KVM, but how do I manage something like serializing
an iommu page table?

I'm replacing all the iommu page table code. It behaves
differently. It supports different things that old kernels don't
understand. iommu page tables have to be under continuous active DMA
during kexec, so they cannot be serialized.

Setting the old stuff as V1 and the new suff as V2 is so easy and is
unconditionally correct. To my point about supporting only stable
branches this achieves it with minimal maintainer effort.

Yes, we could do some comprehensive analysis and try to determine
everything that changed and make micro feature bits, and some thing to
map the current layout to those bits and then hope all of this is
correct.. But that's *a lot* of work, probably will have gaps since it
will never be tested. Seriously, why bother? Explain to me why I
should spend my time on this and who benifits?

IMHO your KVM example has a much clearer answer to that question. It
is UAPI so you have to do it anyhow, and you have a robust open source
ecosystem with alot of VMMs that will consume it. Yeah, I'm on board,
makes sense.

> Have to, or choose to?  I have a very, very hard time believing that
> it's infeasible to define a serialization format that is decoupled
> from kernel internals.

Reduced perfectly everything should become grounded in either HW or
kernel UAPI definitions. Things are not perfect, the kernel has
limitations, it doesn't support every HW feature, stuff leaks
in.. Like my page table example, the new stuff supports ARM CONT, the
old stuff does not. That is kexec ABI breaking and is only happening
because of kernel internal details. In the page table work alone there
are probably ~10-20 micro features like that that would have to be
identified and delt with. Frankly I'm not confident I could even
capture them all.

Also there is the push to make kexec downtime lower. That puts
pressure to just retain kernel things exactly as is, and maybe even
keep kernel memory as-is. ie a sloppy job serializing to get better
performance.

> Only if the relevant subsystem defined a poor save/restore ABI in the first place.

I think KVM is the only subsystem that has save/restore :)

Jason