[PATCH 0/2] Memory and PCI definitions for emulated ATS

[PATCH 0/2] Memory and PCI definitions for emulated ATS

[CLEMENT MATHIEU--DRIF]

This short series adds the 'address type' bit (concept from PCIe) to the
memory attributes and extends the IOMMUAccessFlags enum. This
will be required to implement ATS support for the virtual IOMMUs.

Address type: Field present in the PCIe R/W requests, it allows devices to
tell the IOMMU if the address provided in the request is physical or not.
In other words, it allows the devices to use a physical address obtained
via ATS and to prevent the IOMMU from trying to remap it on the fly.

Additional IOMMU access flags:
    - Execute Requested
    - Privileged Mode Requested
    - Global
    - Untranslated Only (cannot be used with 'Address type = translated')

Clement Mathieu--Drif (2):
  pci: Add a memory attribute for pre-translated DMA operations
  memory: Add permissions in IOMMUAccessFlags

 include/exec/memattrs.h |  3 +++
 include/hw/pci/pci.h    |  9 +++++++++
 include/system/memory.h | 23 +++++++++++++++++++++--
 3 files changed, 33 insertions(+), 2 deletions(-)

-- 
2.49.0

[PATCH 1/2] pci: Add a memory attribute for pre-translated DMA operations

[CLEMENT MATHIEU--DRIF]

The address_type bit will be set to PCI_AT_TRANSLATED by devices that
use cached addresses obtained via ATS.

Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
---
 include/exec/memattrs.h | 3 +++
 include/hw/pci/pci.h    | 9 +++++++++
 2 files changed, 12 insertions(+)

diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
index 8db1d30464..52ee955249 100644
--- a/include/exec/memattrs.h
+++ b/include/exec/memattrs.h
@@ -54,6 +54,9 @@ typedef struct MemTxAttrs {
      */
     unsigned int pid:8;
 
+    /* PCI - IOMMU operations, see PCIAddressType */
+    unsigned int address_type:1;
+
     /*
      * Bus masters which don't specify any attributes will get this
      * (via the MEMTXATTRS_UNSPECIFIED constant), so that we can
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index df3cc7b875..6b7d3ac8a3 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -134,6 +134,15 @@ struct PCIHostDeviceAddress {
     unsigned int function;
 };
 
+/*
+ * Represents the Address Type (AT) field in a PCI request,
+ * see MemTxAttrs.address_type
+ */
+typedef enum PCIAddressType {
+    PCI_AT_UNTRANSLATED = 0, /* Default when no attribute is set */
+    PCI_AT_TRANSLATED = 1,
+} PCIAddressType;
+
 typedef void PCIConfigWriteFunc(PCIDevice *pci_dev,
                                 uint32_t address, uint32_t data, int len);
 typedef uint32_t PCIConfigReadFunc(PCIDevice *pci_dev,
-- 
2.49.0

[PATCH 2/2] memory: Add permissions in IOMMUAccessFlags

[CLEMENT MATHIEU--DRIF]

This will be necessary for devices implementing ATS.
We also define a new macro IOMMU_ACCESS_FLAG_FULL in addition to
IOMMU_ACCESS_FLAG to support more access flags.
IOMMU_ACCESS_FLAG is kept for convenience and backward compatibility.

Here are the flags added (defined by the PCIe 5 specification) :
    - Execute Requested
    - Privileged Mode Requested
    - Global
    - Untranslated Only

IOMMU_ACCESS_FLAG sets the additional flags to 0

Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
---
 include/system/memory.h | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/include/system/memory.h b/include/system/memory.h
index 0848690ea4..5c4cbe838a 100644
--- a/include/system/memory.h
+++ b/include/system/memory.h
@@ -109,15 +109,34 @@ struct MemoryRegionSection {
 
 typedef struct IOMMUTLBEntry IOMMUTLBEntry;
 
-/* See address_space_translate: bit 0 is read, bit 1 is write.  */
+/*
+ * See address_space_translate:
+ *      - bit 0 : read
+ *      - bit 1 : write
+ *      - bit 2 : exec
+ *      - bit 3 : priv
+ *      - bit 4 : global
+ *      - bit 5 : untranslated only
+ */
 typedef enum {
     IOMMU_NONE = 0,
     IOMMU_RO   = 1,
     IOMMU_WO   = 2,
     IOMMU_RW   = 3,
+    IOMMU_EXEC = 4,
+    IOMMU_PRIV = 8,
+    IOMMU_GLOBAL = 16,
+    IOMMU_UNTRANSLATED_ONLY = 32,
 } IOMMUAccessFlags;
 
-#define IOMMU_ACCESS_FLAG(r, w) (((r) ? IOMMU_RO : 0) | ((w) ? IOMMU_WO : 0))
+#define IOMMU_ACCESS_FLAG(r, w)     (((r) ? IOMMU_RO : 0) | \
+                                    ((w) ? IOMMU_WO : 0))
+#define IOMMU_ACCESS_FLAG_FULL(r, w, x, p, g, uo) \
+                                    (IOMMU_ACCESS_FLAG(r, w) | \
+                                    ((x) ? IOMMU_EXEC : 0) | \
+                                    ((p) ? IOMMU_PRIV : 0) | \
+                                    ((g) ? IOMMU_GLOBAL : 0) | \
+                                    ((uo) ? IOMMU_UNTRANSLATED_ONLY : 0))
 
 struct IOMMUTLBEntry {
     AddressSpace    *target_as;
-- 
2.49.0

Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS

[Peter Xu]

On Fri, Jun 20, 2025 at 05:56:49AM +0000, CLEMENT MATHIEU--DRIF wrote:
> This short series adds the 'address type' bit (concept from PCIe) to the
> memory attributes and extends the IOMMUAccessFlags enum. This
> will be required to implement ATS support for the virtual IOMMUs.
> 
> Address type: Field present in the PCIe R/W requests, it allows devices to
> tell the IOMMU if the address provided in the request is physical or not.
> In other words, it allows the devices to use a physical address obtained
> via ATS and to prevent the IOMMU from trying to remap it on the fly.

Two pure questions on the flags, could be relevant to spec:

> 
> Additional IOMMU access flags:
>     - Execute Requested

Does this mean that we can start to put code into DMA regions so that
device can run some day (even if the device may have a core that is totally
different arch v.s. the host's CPUs)?

>     - Privileged Mode Requested
>     - Global
>     - Untranslated Only (cannot be used with 'Address type = translated')

I can understand this with patch 1, but not yet with patch 2.

Patch 1 makes sense to me, IIUC it means the addresses to be used in a pcie
request will be translated addresses which should bypass IOMMU DMAR.

OTOH, patch 2 added it into iotlb access permissions, which I'm not sure
what does it mean.  Perhaps those addresses can only be translated by ATS
pre-translation requests, so that DMA on top of them (in IOVA address
space) will directly fail?

Side note, it might still be more reasonable to put these changes into the
ATS series as the first user of flags.

Thanks,

> 
> Clement Mathieu--Drif (2):
>   pci: Add a memory attribute for pre-translated DMA operations
>   memory: Add permissions in IOMMUAccessFlags
> 
>  include/exec/memattrs.h |  3 +++
>  include/hw/pci/pci.h    |  9 +++++++++
>  include/system/memory.h | 23 +++++++++++++++++++++--
>  3 files changed, 33 insertions(+), 2 deletions(-)
> 
> -- 
> 2.49.0
> 

-- 
Peter Xu

Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS

[CLEMENT MATHIEU--DRIF]

Hi Peter

On 20/06/2025 4:35 pm, Peter Xu wrote:
> Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
> 
> 
> On Fri, Jun 20, 2025 at 05:56:49AM +0000, CLEMENT MATHIEU--DRIF wrote:
>> This short series adds the 'address type' bit (concept from PCIe) to the
>> memory attributes and extends the IOMMUAccessFlags enum. This
>> will be required to implement ATS support for the virtual IOMMUs.
>>
>> Address type: Field present in the PCIe R/W requests, it allows devices to
>> tell the IOMMU if the address provided in the request is physical or not.
>> In other words, it allows the devices to use a physical address obtained
>> via ATS and to prevent the IOMMU from trying to remap it on the fly.
> 
> Two pure questions on the flags, could be relevant to spec:
> 
>>
>> Additional IOMMU access flags:
>>      - Execute Requested
> 
> Does this mean that we can start to put code into DMA regions so that
> device can run some day (even if the device may have a core that is totally
> different arch v.s. the host's 
AFAIU, the spec is so nonrestrictive about this flag that heterogeneous 
arch should not be an issue.

"The definition of what it means for a Function to execute code is 
outside the scope of this specification"

> 
>>      - Privileged Mode Requested
>>      - Global
>>      - Untranslated Only (cannot be used with 'Address type = translated')
> 
> I can understand this with patch 1, but not yet with patch 2.
> 
> Patch 1 makes sense to me, IIUC it means the addresses to be used in a pcie
> request will be translated addresses which should bypass IOMMU DMAR.
> 
> OTOH, patch 2 added it into iotlb access permissions, which I'm not sure
> what does it mean.  Perhaps those addresses can only be translated by ATS
> pre-translation requests, so that DMA on top of them (in IOVA address
> space) will directly fail?

I put this here because the ATS API returns IOMMUTLBEntry structures, 
which contain these flags.

The untranslated-only bit is set in ATS responses to inform the device 
that the requested address cannot be pre-translated and should be 
translated on the fly by the DMA remapping engine. The interrupt range 
commonly falls into this category.

> 
> Side note, it might still be more reasonable to put these changes into the
> ATS series as the first user of flags.

Yes, I can do that.
However, the ATS series will contain ~10/~12 patches, is it a concern?

Thanks
 >cmd

> 
> Thanks,
> 
>>
>> Clement Mathieu--Drif (2):
>>    pci: Add a memory attribute for pre-translated DMA operations
>>    memory: Add permissions in IOMMUAccessFlags
>>
>>   include/exec/memattrs.h |  3 +++
>>   include/hw/pci/pci.h    |  9 +++++++++
>>   include/system/memory.h | 23 +++++++++++++++++++++--
>>   3 files changed, 33 insertions(+), 2 deletions(-)
>>
>> --
>> 2.49.0
>>
> 
> --
> Peter Xu
> 

Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS

[Peter Xu]

On Mon, Jun 23, 2025 at 05:43:06AM +0000, CLEMENT MATHIEU--DRIF wrote:
> Hi Peter
> 
> On 20/06/2025 4:35 pm, Peter Xu wrote:
> > Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
> > 
> > 
> > On Fri, Jun 20, 2025 at 05:56:49AM +0000, CLEMENT MATHIEU--DRIF wrote:
> >> This short series adds the 'address type' bit (concept from PCIe) to the
> >> memory attributes and extends the IOMMUAccessFlags enum. This
> >> will be required to implement ATS support for the virtual IOMMUs.
> >>
> >> Address type: Field present in the PCIe R/W requests, it allows devices to
> >> tell the IOMMU if the address provided in the request is physical or not.
> >> In other words, it allows the devices to use a physical address obtained
> >> via ATS and to prevent the IOMMU from trying to remap it on the fly.
> > 
> > Two pure questions on the flags, could be relevant to spec:
> > 
> >>
> >> Additional IOMMU access flags:
> >>      - Execute Requested
> > 
> > Does this mean that we can start to put code into DMA regions so that
> > device can run some day (even if the device may have a core that is totally
> > different arch v.s. the host's 
> AFAIU, the spec is so nonrestrictive about this flag that heterogeneous 
> arch should not be an issue.
> 
> "The definition of what it means for a Function to execute code is 
> outside the scope of this specification"
> 
> > 
> >>      - Privileged Mode Requested
> >>      - Global
> >>      - Untranslated Only (cannot be used with 'Address type = translated')
> > 
> > I can understand this with patch 1, but not yet with patch 2.
> > 
> > Patch 1 makes sense to me, IIUC it means the addresses to be used in a pcie
> > request will be translated addresses which should bypass IOMMU DMAR.
> > 
> > OTOH, patch 2 added it into iotlb access permissions, which I'm not sure
> > what does it mean.  Perhaps those addresses can only be translated by ATS
> > pre-translation requests, so that DMA on top of them (in IOVA address
> > space) will directly fail?
> 
> I put this here because the ATS API returns IOMMUTLBEntry structures, 
> which contain these flags.
> 
> The untranslated-only bit is set in ATS responses to inform the device 
> that the requested address cannot be pre-translated and should be 
> translated on the fly by the DMA remapping engine. The interrupt range 
> commonly falls into this category.

Ah I see.  Yes this makes sense.

> 
> > 
> > Side note, it might still be more reasonable to put these changes into the
> > ATS series as the first user of flags.
> 
> Yes, I can do that.
> However, the ATS series will contain ~10/~12 patches, is it a concern?

Considering the size of this series, IMHO it should be better to stick with
when they're uesd.

Thanks,

-- 
Peter Xu

Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS

[CLEMENT MATHIEU--DRIF]

On 23/06/2025 3:15 pm, Peter Xu wrote:
> Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
> 
> 
> On Mon, Jun 23, 2025 at 05:43:06AM +0000, CLEMENT MATHIEU--DRIF wrote:
>> Hi Peter
>>
>> On 20/06/2025 4:35 pm, Peter Xu wrote:
>>> Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
>>>
>>>
>>> On Fri, Jun 20, 2025 at 05:56:49AM +0000, CLEMENT MATHIEU--DRIF wrote:
>>>> This short series adds the 'address type' bit (concept from PCIe) to the
>>>> memory attributes and extends the IOMMUAccessFlags enum. This
>>>> will be required to implement ATS support for the virtual IOMMUs.
>>>>
>>>> Address type: Field present in the PCIe R/W requests, it allows devices to
>>>> tell the IOMMU if the address provided in the request is physical or not.
>>>> In other words, it allows the devices to use a physical address obtained
>>>> via ATS and to prevent the IOMMU from trying to remap it on the fly.
>>>
>>> Two pure questions on the flags, could be relevant to spec:
>>>
>>>>
>>>> Additional IOMMU access flags:
>>>>       - Execute Requested
>>>
>>> Does this mean that we can start to put code into DMA regions so that
>>> device can run some day (even if the device may have a core that is totally
>>> different arch v.s. the host's
>> AFAIU, the spec is so nonrestrictive about this flag that heterogeneous
>> arch should not be an issue.
>>
>> "The definition of what it means for a Function to execute code is
>> outside the scope of this specification"
>>
>>>
>>>>       - Privileged Mode Requested
>>>>       - Global
>>>>       - Untranslated Only (cannot be used with 'Address type = translated')
>>>
>>> I can understand this with patch 1, but not yet with patch 2.
>>>
>>> Patch 1 makes sense to me, IIUC it means the addresses to be used in a pcie
>>> request will be translated addresses which should bypass IOMMU DMAR.
>>>
>>> OTOH, patch 2 added it into iotlb access permissions, which I'm not sure
>>> what does it mean.  Perhaps those addresses can only be translated by ATS
>>> pre-translation requests, so that DMA on top of them (in IOVA address
>>> space) will directly fail?
>>
>> I put this here because the ATS API returns IOMMUTLBEntry structures,
>> which contain these flags.
>>
>> The untranslated-only bit is set in ATS responses to inform the device
>> that the requested address cannot be pre-translated and should be
>> translated on the fly by the DMA remapping engine. The interrupt range
>> commonly falls into this category.
> 
> Ah I see.  Yes this makes sense.
> 
>>
>>>
>>> Side note, it might still be more reasonable to put these changes into the
>>> ATS series as the first user of flags.
>>
>> Yes, I can do that.
>> However, the ATS series will contain ~10/~12 patches, is it a concern?
> 
> Considering the size of this series, IMHO it should be better to stick with
> when they're uesd.

Ok, will do!
Thanks Peter

 >cmd

> 
> Thanks,
> 
> --
> Peter Xu
>