Re: [PATCH 00/20] Add SMMUv3 Stage 1 Support for XEN guests

[Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>]

Hi Milan,

Milan Djokic <milan_djokic@epam.com> writes:

> Hello Julien,
>
> On 8/13/25 14:11, Julien Grall wrote:
>> On 13/08/2025 11:04, Milan Djokic wrote:
>>> Hello Julien,
>> Hi Milan,
>> 
>>>
>>> We have prepared a design document and it will be part of the updated
>>> patch series (added in docs/design). I'll also extend cover letter with
>>> details on implementation structure to make review easier.
>> I would suggest to just iterate on the design document for now.
>> 
>>> Following is the design document content which will be provided in
>>> updated patch series:
>>>
>>> Design Proposal: Add SMMUv3 Stage-1 Support for XEN Guests
>>> ==========================================================
>>>
>>> Author: Milan Djokic <milan_djokic@epam.com>
>>> Date:   2025-08-07
>>> Status: Draft
>>>
>>> Introduction
>>> ------------
>>>
>>> The SMMUv3 supports two stages of translation. Each stage of translation
>>> can be independently enabled. An incoming address is logically
>>> translated from VA to IPA in stage 1, then the IPA is input to stage 2
>>> which translates the IPA to the output PA. Stage 1 translation support
>>> is required to provide isolation between different devices within the OS.
>>>
>>> Xen already supports Stage 2 translation but there is no support for
>>> Stage 1 translation. This design proposal outlines the introduction of
>>> Stage-1 SMMUv3 support in Xen for ARM guests.
>>>
>>> Motivation
>>> ----------
>>>
>>> ARM systems utilizing SMMUv3 require Stage-1 address translation to
>>> ensure correct and secure DMA behavior inside guests.
>> Can you clarify what you mean by "correct"? DMA would still work
>> without
>> stage-1.
>
> Correct in terms of working with guest managed I/O space. I'll
> rephrase this statement, it seems ambiguous.
>
>>>
>>> This feature enables:
>>> - Stage-1 translation in guest domain
>>> - Safe device passthrough under secure memory translation
>>>
>>> Design Overview
>>> ---------------
>>>
>>> These changes provide emulated SMMUv3 support:
>>>
>>> - SMMUv3 Stage-1 Translation: stage-1 and nested translation support in
>>>     SMMUv3 driver
>>> - vIOMMU Abstraction: virtual IOMMU framework for guest Stage-1 handling
>> So what are you planning to expose to a guest? Is it one vIOMMU per
>> pIOMMU? Or a single one?
>
> Single vIOMMU model is used in this design.
>
>> Have you considered the pros/cons for both?
>>> - Register/Command Emulation: SMMUv3 register emulation and command
>>>     queue handling
>> 
>
> That's a point for consideration.
> single vIOMMU prevails in terms of less complex implementation and a
> simple guest iommmu model - single vIOMMU node, one interrupt path,
> event queue, single set of trap handlers for emulation, etc.
> Cons for a single vIOMMU model could be less accurate hw
> representation and a potential bottleneck with one emulated queue and
> interrupt path.
> On the other hand, vIOMMU per pIOMMU provides more accurate hw
> modeling and offers better scalability in case of many IOMMUs in the
> system, but this comes with more complex emulation logic and device
> tree, also handling multiple vIOMMUs on guest side.
> IMO, single vIOMMU model seems like a better option mostly because
> it's less complex, easier to maintain and debug. Of course, this
> decision can and should be discussed.
>

Well, I am not sure that this is possible, because of StreamID
allocation. The biggest offender is of course PCI, as each Root PCI
bridge will require own SMMU instance with own StreamID space. But even
without PCI you'll need some mechanism to map vStremID to
<pSMMU, pStreamID>, because there will be overlaps in SID space.


Actually, PCI/vPCI with vSMMU is its own can of worms...

>> For each pSMMU, we have a single command queue that will receive command
>> from all the guests. How do you plan to prevent a guest hogging the
>> command queue?
>> In addition to that, AFAIU, the size of the virtual command queue is
>> fixed by the guest rather than Xen. If a guest is filling up the queue
>> with commands before notifying Xen, how do you plan to ensure we don't
>> spend too much time in Xen (which is not preemptible)?
>> 
>
> We'll have to do a detailed analysis on these scenarios, they are not
> covered by the design (as well as some others which is clear after
> your comments). I'll come back with an updated design.

I think that can be handled akin to hypercall continuation, which is
used in similar places, like P2M code

[...]

-- 
WBR, Volodymyr