[Qemu-devel] About AddressSpace in intel-iommu emulation

[Qemu-devel] About AddressSpace in intel-iommu emulation

[Le Tan]

Hi Paolo,
I am adding intel-iommu emulation to q35 for the GSoC project. I am
confused about AddressSpace and I believe that you can help me. :)
1. For intel-iommu emulation, I have to read the translation
structures from guest memory, that is, the guest will prepare some
tables in memory and write the physical address of them to a register
of intel-iommu, and I need to access those structures. I use
dma_memory_read(&address_space_memory,...) to do this. Is that right?
I am not sure that whether accesses to address_space_memory will be
translated through IOMMU. I think the answer is not, because I see
that cpu_physical_memory_read() also use address_space_memory as
AddressSpace.

2. In my opinion, I have to init a AddressSpace and link it with my
IOMMU MemoryRegion, then the bus uses this AddressSpace to translate
the accesses. Is that right? For q35, how can I register my IOMMU
MemoryRegion to the bus? I see that there is function
pci_setup_iommu() that links a AddressSpace to the bus to translate
accesses to PCI into system memory. Is that related? I think q35
should maintain a bus AddressSpace, but I can't find it.
What do you think?

Thanks very much!

Regards,
Le Tan

Re: [Qemu-devel] About AddressSpace in intel-iommu emulation

[Paolo Bonzini]

Il 26/06/2014 16:01, Le Tan ha scritto:
> Hi Paolo,
> I am adding intel-iommu emulation to q35 for the GSoC project. I am
> confused about AddressSpace and I believe that you can help me. :)
> 1. For intel-iommu emulation, I have to read the translation
> structures from guest memory, that is, the guest will prepare some
> tables in memory and write the physical address of them to a register
> of intel-iommu, and I need to access those structures. I use
> dma_memory_read(&address_space_memory,...) to do this. Is that right?
> I am not sure that whether accesses to address_space_memory will be
> translated through IOMMU. I think the answer is not, because I see
> that cpu_physical_memory_read() also use address_space_memory as
> AddressSpace.

Correct.

> 2. In my opinion, I have to init a AddressSpace and link it with my
> IOMMU MemoryRegion, then the bus uses this AddressSpace to translate
> the accesses. Is that right? For q35, how can I register my IOMMU
> MemoryRegion to the bus? I see that there is function
> pci_setup_iommu() that links a AddressSpace to the bus to translate
> accesses to PCI into system memory. Is that related? I think q35
> should maintain a bus AddressSpace, but I can't find it.
> What do you think?

Right now, the q35 PCI host does not define an iommu_fn, so the default 
DMA address space is used by pci_device_iommu_address_space.  This is 
just address_space_memory.

The iommu_fn is set with pci_setup_iommu.  Commit ae74bbe (apb: 
implement IOMMU translation for PCI host bridge, 2014-05-28) provides an 
example of how to prepare an IOMMU memory region, add it to an address 
space, and return that address space from an iommu_fn.

Thanks,

Paolo

> Thanks very much!
>
> Regards,
> Le Tan
>

Re: [Qemu-devel] About AddressSpace in intel-iommu emulation

[Le Tan]

2014-06-26 22:05 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
> Il 26/06/2014 16:01, Le Tan ha scritto:
>
>> Hi Paolo,
>> I am adding intel-iommu emulation to q35 for the GSoC project. I am
>> confused about AddressSpace and I believe that you can help me. :)
>> 1. For intel-iommu emulation, I have to read the translation
>> structures from guest memory, that is, the guest will prepare some
>> tables in memory and write the physical address of them to a register
>> of intel-iommu, and I need to access those structures. I use
>> dma_memory_read(&address_space_memory,...) to do this. Is that right?
>> I am not sure that whether accesses to address_space_memory will be
>> translated through IOMMU. I think the answer is not, because I see
>> that cpu_physical_memory_read() also use address_space_memory as
>> AddressSpace.
>
>
> Correct.
>
>
>> 2. In my opinion, I have to init a AddressSpace and link it with my
>> IOMMU MemoryRegion, then the bus uses this AddressSpace to translate
>> the accesses. Is that right? For q35, how can I register my IOMMU
>> MemoryRegion to the bus? I see that there is function
>> pci_setup_iommu() that links a AddressSpace to the bus to translate
>> accesses to PCI into system memory. Is that related? I think q35
>> should maintain a bus AddressSpace, but I can't find it.
>> What do you think?
>
>
> Right now, the q35 PCI host does not define an iommu_fn, so the default DMA
> address space is used by pci_device_iommu_address_space.  This is just
> address_space_memory.
>
> The iommu_fn is set with pci_setup_iommu.  Commit ae74bbe (apb: implement
> IOMMU translation for PCI host bridge, 2014-05-28) provides an example of
> how to prepare an IOMMU memory region, add it to an address space, and
> return that address space from an iommu_fn.
>
> Thanks,

Thanks very much! I saw the example of apb and typoon before and was
confused that why there is no such thing in q35. And it is clear now.
Thanks very much! :)
ps: Send this email again because I forgot to reply to all in last
email. Sorry. :)

Regards,
Le

> Paolo
>
>
>> Thanks very much!
>>
>> Regards,
>> Le Tan
>>
>

Re: [Qemu-devel] About AddressSpace in intel-iommu emulation

[Le Tan]

2014-06-26 22:05 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
> Il 26/06/2014 16:01, Le Tan ha scritto:
>
>> Hi Paolo,
>> I am adding intel-iommu emulation to q35 for the GSoC project. I am
>> confused about AddressSpace and I believe that you can help me. :)
>> 1. For intel-iommu emulation, I have to read the translation
>> structures from guest memory, that is, the guest will prepare some
>> tables in memory and write the physical address of them to a register
>> of intel-iommu, and I need to access those structures. I use
>> dma_memory_read(&address_space_memory,...) to do this. Is that right?
>> I am not sure that whether accesses to address_space_memory will be
>> translated through IOMMU. I think the answer is not, because I see
>> that cpu_physical_memory_read() also use address_space_memory as
>> AddressSpace.
>
>
> Correct.
>
>
>> 2. In my opinion, I have to init a AddressSpace and link it with my
>> IOMMU MemoryRegion, then the bus uses this AddressSpace to translate
>> the accesses. Is that right? For q35, how can I register my IOMMU
>> MemoryRegion to the bus? I see that there is function
>> pci_setup_iommu() that links a AddressSpace to the bus to translate
>> accesses to PCI into system memory. Is that related? I think q35
>> should maintain a bus AddressSpace, but I can't find it.
>> What do you think?
>
>
> Right now, the q35 PCI host does not define an iommu_fn, so the default DMA
> address space is used by pci_device_iommu_address_space.  This is just
> address_space_memory.
>
> The iommu_fn is set with pci_setup_iommu.  Commit ae74bbe (apb: implement
> IOMMU translation for PCI host bridge, 2014-05-28) provides an example of
> how to prepare an IOMMU memory region, add it to an address space, and
> return that address space from an iommu_fn.

Hi Paolo,
I have added the address space to q35 and the translate function of
intel_iommu is called. :) However, I still have some questions here.
1. In struct IOMMUTLBEntry, I think the addr_mask field should be the
mask of the page offset, right? But I see different usages of this
field. In spapr_tce_translate_iommu(), the addr_mask field is assigned
with the mask of the page offset. However, in pbm_translate_iommu(),
in the passthrough case, the addr_mask field seems to be assigned the
mask of the page number. Is there any problem here?
2. For q35, how to identify origination of DMA requests? The VT-d
manual says we should use source-id(for PCI-Express devices, it is
requester identifier) to map devices to domains. What is the related
part in QEMU? Where can I get the source-id of a DMA request?

Thanks very much!

Le


> Thanks,
>
> Paolo
>
>
>> Thanks very much!
>>
>> Regards,
>> Le Tan
>>
>

Re: [Qemu-devel] About AddressSpace in intel-iommu emulation

[Paolo Bonzini]

Il 27/06/2014 04:08, Le Tan ha scritto:
> 1. In struct IOMMUTLBEntry, I think the addr_mask field should be the
> mask of the page offset, right? But I see different usages of this
> field. In spapr_tce_translate_iommu(), the addr_mask field is assigned
> with the mask of the page offset. However, in pbm_translate_iommu(),
> in the passthrough case, the addr_mask field seems to be assigned the
> mask of the page number. Is there any problem here?

The intended usage is the one of spapr_tce_translate_iommu().  In 
practice it doesn't matter, both work.

> 2. For q35, how to identify origination of DMA requests? The VT-d
> manual says we should use source-id(for PCI-Express devices, it is
> requester identifier) to map devices to domains. What is the related
> part in QEMU? Where can I get the source-id of a DMA request?

You need to create a different AddressSpace for each PCI bus or device.

Paolo

Re: [Qemu-devel] About AddressSpace in intel-iommu emulation

[Le Tan]

2014-06-27 12:55 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
> Il 27/06/2014 04:08, Le Tan ha scritto:
>
>> 1. In struct IOMMUTLBEntry, I think the addr_mask field should be the
>> mask of the page offset, right? But I see different usages of this
>> field. In spapr_tce_translate_iommu(), the addr_mask field is assigned
>> with the mask of the page offset. However, in pbm_translate_iommu(),
>> in the passthrough case, the addr_mask field seems to be assigned the
>> mask of the page number. Is there any problem here?
>
>
> The intended usage is the one of spapr_tce_translate_iommu().  In practice
> it doesn't matter, both work.
>
>
>> 2. For q35, how to identify origination of DMA requests? The VT-d
>> manual says we should use source-id(for PCI-Express devices, it is
>> requester identifier) to map devices to domains. What is the related
>> part in QEMU? Where can I get the source-id of a DMA request?
>
>
> You need to create a different AddressSpace for each PCI bus or device.

How to create a different AddressSpace for each device? I thought a
AddressSpace just belongs to a PCI bus before. The paging structures
for different functions of the same device can also be different, too.
So maybe we should create a different AddressSpace for each function?
How to achieve it? Could you give me some more hints or is there any
existing example in QEMU?
Thanks very much!

Le

> Paolo

Re: [Qemu-devel] About AddressSpace in intel-iommu emulation

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1811 bytes --]

On 2014-06-27 07:46, Le Tan wrote:
> 2014-06-27 12:55 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
>> Il 27/06/2014 04:08, Le Tan ha scritto:
>>
>>> 1. In struct IOMMUTLBEntry, I think the addr_mask field should be the
>>> mask of the page offset, right? But I see different usages of this
>>> field. In spapr_tce_translate_iommu(), the addr_mask field is assigned
>>> with the mask of the page offset. However, in pbm_translate_iommu(),
>>> in the passthrough case, the addr_mask field seems to be assigned the
>>> mask of the page number. Is there any problem here?
>>
>>
>> The intended usage is the one of spapr_tce_translate_iommu().  In practice
>> it doesn't matter, both work.
>>
>>
>>> 2. For q35, how to identify origination of DMA requests? The VT-d
>>> manual says we should use source-id(for PCI-Express devices, it is
>>> requester identifier) to map devices to domains. What is the related
>>> part in QEMU? Where can I get the source-id of a DMA request?
>>
>>
>> You need to create a different AddressSpace for each PCI bus or device.
> 
> How to create a different AddressSpace for each device? I thought a
> AddressSpace just belongs to a PCI bus before. The paging structures
> for different functions of the same device can also be different, too.
> So maybe we should create a different AddressSpace for each function?
> How to achieve it? Could you give me some more hints or is there any
> existing example in QEMU?

I would suggest to study the apb IOMMU implementation Paolo referenced
and the PCI layer functions used by that code. Specifically,
pci_setup_iommu takes a callback that is supposed to return an address
space to be used for a particular device. For apb, it's the same for all
devices on a bus, but that's not required...

Jan



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: [Qemu-devel] About AddressSpace in intel-iommu emulation

[Le Tan]

2014-06-27 17:55 GMT+08:00 Jan Kiszka <jan.kiszka@web.de>:
> On 2014-06-27 07:46, Le Tan wrote:
>> 2014-06-27 12:55 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
>>> Il 27/06/2014 04:08, Le Tan ha scritto:
>>>
>>>> 1. In struct IOMMUTLBEntry, I think the addr_mask field should be the
>>>> mask of the page offset, right? But I see different usages of this
>>>> field. In spapr_tce_translate_iommu(), the addr_mask field is assigned
>>>> with the mask of the page offset. However, in pbm_translate_iommu(),
>>>> in the passthrough case, the addr_mask field seems to be assigned the
>>>> mask of the page number. Is there any problem here?
>>>
>>>
>>> The intended usage is the one of spapr_tce_translate_iommu().  In practice
>>> it doesn't matter, both work.
>>>
>>>
>>>> 2. For q35, how to identify origination of DMA requests? The VT-d
>>>> manual says we should use source-id(for PCI-Express devices, it is
>>>> requester identifier) to map devices to domains. What is the related
>>>> part in QEMU? Where can I get the source-id of a DMA request?
>>>
>>>
>>> You need to create a different AddressSpace for each PCI bus or device.
>>
>> How to create a different AddressSpace for each device? I thought a
>> AddressSpace just belongs to a PCI bus before. The paging structures
>> for different functions of the same device can also be different, too.
>> So maybe we should create a different AddressSpace for each function?
>> How to achieve it? Could you give me some more hints or is there any
>> existing example in QEMU?
>
> I would suggest to study the apb IOMMU implementation Paolo referenced
> and the PCI layer functions used by that code. Specifically,
> pci_setup_iommu takes a callback that is supposed to return an address
> space to be used for a particular device. For apb, it's the same for all
> devices on a bus, but that's not required...

Yeah, I notice that the third parameter passed to bus->iommu_fn() is
the key. That is the dev->devfn, which is the combination of #device
and #function. I was so careless that I didn't noticed this field. I
think I am getting close to the answer. I will dive into this later.
:）
Thanks very much!

Regards,
Le

>
> Jan
>
>