Xen Summit 2025 - Design Discussion Notes - Xen ABI

["Alexander M. Merritt" <alexander@edera.dev>]

Hi all, it was requested I send the notes I took during the design discussion on the ABI / APIs to the list.

Normally I keep this as personal notes, so there may be errors (esp if I did not hear correctly), so please feel free to correct or expand. Details may be missing where I am unaware of the history behind something.

-Alex Merritt




Design Discussion: Xen ABIs and APIs

- Chris on remote: Andrew has been and wants to work on a new ABI
- Andrew: put together a collection of documents to understand what we have
to work with, what we want to improve, before starting the work on any
design or iterations on interfaces we currently have
- link to document in the design session
        ‐ https://design-sessions.xenproject.org/uid/discussion/disc_3IEQbyaCTkqLf2fFzoze/view
- number of things we have been aware of for a while
- some attempts to address them on the list
- one problem: if you only try to fix one of them, it brings in discussion of
fixing many other items
- everyone has opinion on what the end result will look like
- existing designs only fix subsets, not the whole thing
- we want to address all the problems from the start, before deciding on a plan
to fix them
- enumerate the ABIs and APIs that currently exist
        ‐ problems not apparent if you just think about this
        ‐ many folks think this is just the hypercalls
        ‐ there is the enumeration information
        ‐ xen has many bugs - originally monorepo with xen, linux, qemu, BSDs,
        bochs, ... with “make world” you got a system. All guests were
        required to have event channel - no discovery exists because they all
        had it
        ‐ grant table v2, migrate old version of xen to new, exercise new code
        paths, then kernel crashed
        ‐ initial state of vcpus - many folks don’t think about them, but what
        xen presents, we have bugs describing those via the hypercalls we use
        ‐ the hypercalls themselves -- 46? -- half of them specific for PV guests
                - x86 HVM / ARM HVM are only a small fraction of the total
                hypercalls that exist
- the reason the hypercalls look like this now, Xen started with pv guests on
x86, a VAS system made sense
        ‐ when HVM guests came along, we have hacks fitting PV guests into HVM
        ‐ Xen has to walk the page tables of the guest just to get the
        information it needs, you cannot do that in encrypted VMs by design
        ‐ need to change the way we deal with pointers in the API
- evtchn send, pass pointer information on the stack
        ‐ get interrupt for someone else!
- look over all APIs and ABIs that exist because they have different problems in
different areas
- XenServer cares most about right now host UEFI secure boot
        ‐ new priv boundary that does not exist previously
        ‐ admin with root cannot (should not) violate security boundary, cannot
        read/write arbitrary memroy
        ‐ hypercalls: open /dev/xen/privcmd and pointers into user space
        memory, nothing stops passing kernel pointer memory
                - giant privilege escalation hole in UEFI secure boot
                - root user space is not priv enough to execute arbitrary code
        ‐ all problems compound, thus we want to look at all of them before we
        start figuring out what to do
- another example: being based on x86 originally, large hypercalls have a shift
by 12, assume 4k pages, problem with ARM wanting 64k page tables
        ‐ event the data layout wants to change
- if you change the version of Xen, you break the user space (library versions)
        ‐ was intentional choice early on, doesn’t scale
        ‐ get rid of unstable APIs -- killing xen
- security hotfix - recompile QEMU
        ‐ ABI rules say any change in hypervisor, thus rebuilding user space,
        and QEMU -- anything that links against the xen packages!
- Bertrand: look at problem yesterday: how we create and configure a guest,
coherency to reach dom0less
        ‐ twice code to create a guest, duplicated code
        ‐ duplicate configuration format
        ‐ if we modify ABI between dom0 and Xen, need to look at have
        coherent format so we can reuse the same code
- Alex M: can we hide hypercalls via libraries?
        ‐ yes but currently the versions for a break
        ‐ definitely an option forward
        ‐ still doesn’t solve the issue, because other libraries in other languages
        won’t be shielded from unstable ABIs
- Jan: both knowing what to do and where we go is useful
        ‐ Andrew: have to have broad idea where to go....
- Jan: carrying out hypercall is independent of the mechansim we define
        ‐ Andrew: still needs backwards compatibility
        ‐ Andrew: use higher op numbers
- Alex M: is our problem unique to us?
        ‐ Andrew: we have enough corner cases that yes
        ‐ Bertrand: PV guests require a large number of hypercalls
        ‐ Jan: keep VA for PV hypercalls
- Rich on call: work together with Chris to write down something difficult in
scope
        ‐ any work written down, useful for folks on other side where we may
        encounter failures
        ‐ newcomers: xen forked by HP (?)
        ‐ everyone tried to narrow to verticle markets, focus on specific markets
        ‐ Xen: is last entity standing, still trying to pull all stakeholders together,
        but not sure how long it will last
        ‐ if collapses: accidental or intentional interoperability, carve out the
        pieces so that the ppl at table today have a chance to know what
        results from it
        ‐ what will last longest: certified entities that have long lifecycles,
        decades or more
        ‐ certified snapshots will become longest lived design choices
- Andrew: shared info page
        ‐ layout was done with unsigned longs which changed sizes
        ‐ layout of the shared info page changes
        ‐ different vcpus can be in different modes at a time
        ‐ we cache the mode of the cpu at the point which it makes one of two
        types of hypercalls
- another design session tomorrow