Different type iommus integrated in a SoC

Different type iommus integrated in a SoC

[xxm]

[-- Attachment #1.1: Type: text/plain, Size: 412 bytes --]

Hi all,

I have a SoC integrate with two different types of iommus, one is ARM SMMU, serves the PCIe/SATA/USB,
the others are vendor specific iommus, serves display device and multimedia device.

In the current linux kernel, the iommu framework seems only support one type iommu at runtime, if enable both types iommu, only one type can work.
Is there any way to support this kind of SoC? 


Simon



[-- Attachment #1.2: Type: text/html, Size: 1169 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: Different type iommus integrated in a SoC

[Robin Murphy]

On 2021-05-27 03:37, xxm@rock-chips.com wrote:
> Hi all,
> 
> I have a SoC integrate with two different types of iommus, one is ARM SMMU, serves the PCIe/SATA/USB,
> the others are vendor specific iommus, serves display device and multimedia device.
> 
> In the current linux kernel, the iommu framework seems only support one type iommu at runtime, if enable both types iommu, only one type can work.
> Is there any way to support this kind of SoC?

Hooray! I've been forecasting this for years, but the cases we regularly 
hit with internal FPGA prototyping (nor the secret unused MMU-400 I 
found on RK3288) have never really been a strong enough argument to 
stand behind.

Based on what I remember from looking into this a few years ago, 
converting *most* of the API to per-device ops (now via dev->iommu) is 
trivial; the main challenge will be getting the per-device data 
bootstrapped in iommu_probe_device(), which would probably need to rely 
on the fwspec and/or list of registered IOMMU instances.

The other notable thing which will need to change is the domain 
allocation interface, but in practice I think everyone who calls 
iommu_domain_alloc() today is in fact doing so for a specific device, so 
I don't think it's as big a problem as it might first appear.

Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: Different type iommus integrated in a SoC

[Robin Murphy]

On 2021-05-27 03:37, xxm@rock-chips.com wrote:
> Hi all,
> 
> I have a SoC integrate with two different types of iommus, one is ARM SMMU, serves the PCIe/SATA/USB,
> the others are vendor specific iommus, serves display device and multimedia device.
> 
> In the current linux kernel, the iommu framework seems only support one type iommu at runtime, if enable both types iommu, only one type can work.
> Is there any way to support this kind of SoC?

Hooray! I've been forecasting this for years, but the cases we regularly 
hit with internal FPGA prototyping (nor the secret unused MMU-400 I 
found on RK3288) have never really been a strong enough argument to 
stand behind.

Based on what I remember from looking into this a few years ago, 
converting *most* of the API to per-device ops (now via dev->iommu) is 
trivial; the main challenge will be getting the per-device data 
bootstrapped in iommu_probe_device(), which would probably need to rely 
on the fwspec and/or list of registered IOMMU instances.

The other notable thing which will need to change is the domain 
allocation interface, but in practice I think everyone who calls 
iommu_domain_alloc() today is in fact doing so for a specific device, so 
I don't think it's as big a problem as it might first appear.

Robin.

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: Different type iommus integrated in a SoC

[Peter Geis]

On Thu, Jun 3, 2021 at 8:07 AM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-05-27 03:37, xxm@rock-chips.com wrote:
> > Hi all,
> >
> > I have a SoC integrate with two different types of iommus, one is ARM SMMU, serves the PCIe/SATA/USB,
> > the others are vendor specific iommus, serves display device and multimedia device.
> >
> > In the current linux kernel, the iommu framework seems only support one type iommu at runtime, if enable both types iommu, only one type can work.
> > Is there any way to support this kind of SoC?
>
> Hooray! I've been forecasting this for years, but the cases we regularly
> hit with internal FPGA prototyping (nor the secret unused MMU-400 I
> found on RK3288) have never really been a strong enough argument to
> stand behind.
>
> Based on what I remember from looking into this a few years ago,
> converting *most* of the API to per-device ops (now via dev->iommu) is
> trivial; the main challenge will be getting the per-device data
> bootstrapped in iommu_probe_device(), which would probably need to rely
> on the fwspec and/or list of registered IOMMU instances.
>
> The other notable thing which will need to change is the domain
> allocation interface, but in practice I think everyone who calls
> iommu_domain_alloc() today is in fact doing so for a specific device, so
> I don't think it's as big a problem as it might first appear.
>
> Robin.
>

Good Morning Robin,

I think the Tegra group would also be interested in this work.
AFAIK they have the smmu and the tegra gart and have been trying to
figure out the runtime handover from the bootloader to the kernel
without smashing everything and starting over.

Thanks,
Peter Geis
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: Different type iommus integrated in a SoC

[Peter Geis]

On Thu, Jun 3, 2021 at 8:07 AM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-05-27 03:37, xxm@rock-chips.com wrote:
> > Hi all,
> >
> > I have a SoC integrate with two different types of iommus, one is ARM SMMU, serves the PCIe/SATA/USB,
> > the others are vendor specific iommus, serves display device and multimedia device.
> >
> > In the current linux kernel, the iommu framework seems only support one type iommu at runtime, if enable both types iommu, only one type can work.
> > Is there any way to support this kind of SoC?
>
> Hooray! I've been forecasting this for years, but the cases we regularly
> hit with internal FPGA prototyping (nor the secret unused MMU-400 I
> found on RK3288) have never really been a strong enough argument to
> stand behind.
>
> Based on what I remember from looking into this a few years ago,
> converting *most* of the API to per-device ops (now via dev->iommu) is
> trivial; the main challenge will be getting the per-device data
> bootstrapped in iommu_probe_device(), which would probably need to rely
> on the fwspec and/or list of registered IOMMU instances.
>
> The other notable thing which will need to change is the domain
> allocation interface, but in practice I think everyone who calls
> iommu_domain_alloc() today is in fact doing so for a specific device, so
> I don't think it's as big a problem as it might first appear.
>
> Robin.
>

Good Morning Robin,

I think the Tegra group would also be interested in this work.
AFAIK they have the smmu and the tegra gart and have been trying to
figure out the runtime handover from the bootloader to the kernel
without smashing everything and starting over.

Thanks,
Peter Geis

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: Different type iommus integrated in a SoC

[Robin Murphy]

On 2021-06-03 13:24, Peter Geis wrote:
> On Thu, Jun 3, 2021 at 8:07 AM Robin Murphy <robin.murphy@arm.com> wrote:
>>
>> On 2021-05-27 03:37, xxm@rock-chips.com wrote:
>>> Hi all,
>>>
>>> I have a SoC integrate with two different types of iommus, one is ARM SMMU, serves the PCIe/SATA/USB,
>>> the others are vendor specific iommus, serves display device and multimedia device.
>>>
>>> In the current linux kernel, the iommu framework seems only support one type iommu at runtime, if enable both types iommu, only one type can work.
>>> Is there any way to support this kind of SoC?
>>
>> Hooray! I've been forecasting this for years, but the cases we regularly
>> hit with internal FPGA prototyping (nor the secret unused MMU-400 I
>> found on RK3288) have never really been a strong enough argument to
>> stand behind.
>>
>> Based on what I remember from looking into this a few years ago,
>> converting *most* of the API to per-device ops (now via dev->iommu) is
>> trivial; the main challenge will be getting the per-device data
>> bootstrapped in iommu_probe_device(), which would probably need to rely
>> on the fwspec and/or list of registered IOMMU instances.
>>
>> The other notable thing which will need to change is the domain
>> allocation interface, but in practice I think everyone who calls
>> iommu_domain_alloc() today is in fact doing so for a specific device, so
>> I don't think it's as big a problem as it might first appear.
>>
>> Robin.
>>
> 
> Good Morning Robin,
> 
> I think the Tegra group would also be interested in this work.
> AFAIK they have the smmu and the tegra gart and have been trying to
> figure out the runtime handover from the bootloader to the kernel
> without smashing everything and starting over.

No, handoff of live DMA from the bootlader is an entirely unrelated 
issue, and there are already several patchsets in flight to address 
various parts of that. My understanding of Tegras is that they *either* 
use tegra-gart, tegra-smmu, or arm-smmu depending on the SoC generation, 
but they aren't mixed within any single SoC.

Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: Different type iommus integrated in a SoC

[Robin Murphy]

On 2021-06-03 13:24, Peter Geis wrote:
> On Thu, Jun 3, 2021 at 8:07 AM Robin Murphy <robin.murphy@arm.com> wrote:
>>
>> On 2021-05-27 03:37, xxm@rock-chips.com wrote:
>>> Hi all,
>>>
>>> I have a SoC integrate with two different types of iommus, one is ARM SMMU, serves the PCIe/SATA/USB,
>>> the others are vendor specific iommus, serves display device and multimedia device.
>>>
>>> In the current linux kernel, the iommu framework seems only support one type iommu at runtime, if enable both types iommu, only one type can work.
>>> Is there any way to support this kind of SoC?
>>
>> Hooray! I've been forecasting this for years, but the cases we regularly
>> hit with internal FPGA prototyping (nor the secret unused MMU-400 I
>> found on RK3288) have never really been a strong enough argument to
>> stand behind.
>>
>> Based on what I remember from looking into this a few years ago,
>> converting *most* of the API to per-device ops (now via dev->iommu) is
>> trivial; the main challenge will be getting the per-device data
>> bootstrapped in iommu_probe_device(), which would probably need to rely
>> on the fwspec and/or list of registered IOMMU instances.
>>
>> The other notable thing which will need to change is the domain
>> allocation interface, but in practice I think everyone who calls
>> iommu_domain_alloc() today is in fact doing so for a specific device, so
>> I don't think it's as big a problem as it might first appear.
>>
>> Robin.
>>
> 
> Good Morning Robin,
> 
> I think the Tegra group would also be interested in this work.
> AFAIK they have the smmu and the tegra gart and have been trying to
> figure out the runtime handover from the bootloader to the kernel
> without smashing everything and starting over.

No, handoff of live DMA from the bootlader is an entirely unrelated 
issue, and there are already several patchsets in flight to address 
various parts of that. My understanding of Tegras is that they *either* 
use tegra-gart, tegra-smmu, or arm-smmu depending on the SoC generation, 
but they aren't mixed within any single SoC.

Robin.

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: Different type iommus integrated in a SoC

[joro]

On Thu, Jun 03, 2021 at 01:05:43PM +0100, Robin Murphy wrote:
> Hooray! I've been forecasting this for years, but the cases we regularly hit
> with internal FPGA prototyping (nor the secret unused MMU-400 I found on
> RK3288) have never really been a strong enough argument to stand behind.
> 
> Based on what I remember from looking into this a few years ago, converting
> *most* of the API to per-device ops (now via dev->iommu) is trivial; the
> main challenge will be getting the per-device data bootstrapped in
> iommu_probe_device(), which would probably need to rely on the fwspec and/or
> list of registered IOMMU instances.
> 
> The other notable thing which will need to change is the domain allocation
> interface, but in practice I think everyone who calls iommu_domain_alloc()
> today is in fact doing so for a specific device, so I don't think it's as
> big a problem as it might first appear.

Yeah, I think for that we have to give up on the promise that a domain
can be assigned to _any_ device. But this promise doesn't even hold true
now when there are several IOMMU of the same type but with different
feature sets in a system.

So I happily review patches enabling the Multi-IOMMU SOCs :)

Regards,

	Joerg
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: Different type iommus integrated in a SoC

[joro]

On Thu, Jun 03, 2021 at 01:05:43PM +0100, Robin Murphy wrote:
> Hooray! I've been forecasting this for years, but the cases we regularly hit
> with internal FPGA prototyping (nor the secret unused MMU-400 I found on
> RK3288) have never really been a strong enough argument to stand behind.
> 
> Based on what I remember from looking into this a few years ago, converting
> *most* of the API to per-device ops (now via dev->iommu) is trivial; the
> main challenge will be getting the per-device data bootstrapped in
> iommu_probe_device(), which would probably need to rely on the fwspec and/or
> list of registered IOMMU instances.
> 
> The other notable thing which will need to change is the domain allocation
> interface, but in practice I think everyone who calls iommu_domain_alloc()
> today is in fact doing so for a specific device, so I don't think it's as
> big a problem as it might first appear.

Yeah, I think for that we have to give up on the promise that a domain
can be assigned to _any_ device. But this promise doesn't even hold true
now when there are several IOMMU of the same type but with different
feature sets in a system.

So I happily review patches enabling the Multi-IOMMU SOCs :)

Regards,

	Joerg

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip