Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Thierry Reding]

[-- Attachment #1.1: Type: text/plain, Size: 2729 bytes --]

On Wed, Apr 24, 2019 at 12:52:31PM +0100, Will Deacon wrote:
> On Wed, Apr 24, 2019 at 01:36:58PM +0200, Marc Gonzalez wrote:
> > On 04/04/2019 17:00, Will Deacon wrote:
> > 
> > > On Fri, Mar 01, 2019 at 11:20:17AM -0800, Douglas Anderson wrote:
> > >
> > >> If you're bisecting why your peripherals stopped working, it's
> > >> probably this CL.  Specifically if you see this in your dmesg:
> > >>   Unexpected global fault, this could be serious
> > >> ...then it's almost certainly this CL.
> > >>
> > >> Running your IOMMU-enabled peripherals with the IOMMU in bypass mode
> > >> is insecure and effectively disables the protection they provide.
> > >> There are few reasons to allow unmatched stream bypass, and even fewer
> > >> good ones.
> > >>
> > >> This patch starts the transition over to make it much harder to run
> > >> your system insecurely.  Expected steps:
> > >>
> > >> 1. By default disable bypass (so anyone insecure will notice) but make
> > >>    it easy for someone to re-enable bypass with just a KConfig change.
> > >>    That's this patch.
> > >>
> > >> 2. After people have had a little time to come to grips with the fact
> > >>    that they need to set their IOMMUs properly and have had time to
> > >>    dig into how to do this, the KConfig will be eliminated and bypass
> > >>    will simply be disabled.  Folks who are truly upset and still
> > >>    haven't fixed their system can either figure out how to add
> > >>    'arm-smmu.disable_bypass=n' to their command line or revert the
> > >>    patch in their own private kernel.  Of course these folks will be
> > >>    less secure.
> > >>
> > >> Suggested-by: Robin Murphy <robin.murphy@arm.com>
> > >> Signed-off-by: Douglas Anderson <dianders@chromium.org>
> > >> ---
> > >>
> > >> Changes in v2:
> > >> - Flipped default to 'yes' and changed comments a lot.
> > >>
> > >>  drivers/iommu/Kconfig    | 25 +++++++++++++++++++++++++
> > >>  drivers/iommu/arm-smmu.c |  3 ++-
> > >>  2 files changed, 27 insertions(+), 1 deletion(-)
> > > 
> > > Cheers, I'll pick this one up for 5.2.
> > 
> > Hello Will,
> > 
> > You haven't pushed this patch out to linux-next AFAICT.
> > 
> > Is that expected?
> 
> It's on my branch for Joerg:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=for-joerg/arm-smmu/updates
> 
> which I'll send to him today. My SMMU stuff doesn't go directly into -next.

This made it to linux-next yesterday (less than a week before the merge
window opens) and deliberately breaks existing configurations. That's a
little rude.

At least give people a fair heads-up and a chance to fix things before
you start break things.

Thierry

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Will Deacon]

On Thu, May 02, 2019 at 12:59:12PM +0200, Thierry Reding wrote:
> On Wed, Apr 24, 2019 at 12:52:31PM +0100, Will Deacon wrote:
> > On Wed, Apr 24, 2019 at 01:36:58PM +0200, Marc Gonzalez wrote:
> > > On 04/04/2019 17:00, Will Deacon wrote:
> > > 
> > > > On Fri, Mar 01, 2019 at 11:20:17AM -0800, Douglas Anderson wrote:
> > > >
> > > >> If you're bisecting why your peripherals stopped working, it's
> > > >> probably this CL.  Specifically if you see this in your dmesg:
> > > >>   Unexpected global fault, this could be serious
> > > >> ...then it's almost certainly this CL.
> > > >>
> > > >> Running your IOMMU-enabled peripherals with the IOMMU in bypass mode
> > > >> is insecure and effectively disables the protection they provide.
> > > >> There are few reasons to allow unmatched stream bypass, and even fewer
> > > >> good ones.
> > > >>
> > > >> This patch starts the transition over to make it much harder to run
> > > >> your system insecurely.  Expected steps:
> > > >>
> > > >> 1. By default disable bypass (so anyone insecure will notice) but make
> > > >>    it easy for someone to re-enable bypass with just a KConfig change.
> > > >>    That's this patch.
> > > >>
> > > >> 2. After people have had a little time to come to grips with the fact
> > > >>    that they need to set their IOMMUs properly and have had time to
> > > >>    dig into how to do this, the KConfig will be eliminated and bypass
> > > >>    will simply be disabled.  Folks who are truly upset and still
> > > >>    haven't fixed their system can either figure out how to add
> > > >>    'arm-smmu.disable_bypass=n' to their command line or revert the
> > > >>    patch in their own private kernel.  Of course these folks will be
> > > >>    less secure.
> > > >>
> > > >> Suggested-by: Robin Murphy <robin.murphy@arm.com>
> > > >> Signed-off-by: Douglas Anderson <dianders@chromium.org>
> > > >> ---
> > > >>
> > > >> Changes in v2:
> > > >> - Flipped default to 'yes' and changed comments a lot.
> > > >>
> > > >>  drivers/iommu/Kconfig    | 25 +++++++++++++++++++++++++
> > > >>  drivers/iommu/arm-smmu.c |  3 ++-
> > > >>  2 files changed, 27 insertions(+), 1 deletion(-)
> > > > 
> > > > Cheers, I'll pick this one up for 5.2.
> > > 
> > > Hello Will,
> > > 
> > > You haven't pushed this patch out to linux-next AFAICT.
> > > 
> > > Is that expected?
> > 
> > It's on my branch for Joerg:
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=for-joerg/arm-smmu/updates
> > 
> > which I'll send to him today. My SMMU stuff doesn't go directly into -next.
> 
> This made it to linux-next yesterday (less than a week before the merge
> window opens) and deliberately breaks existing configurations. That's a
> little rude.
> 
> At least give people a fair heads-up and a chance to fix things before
> you start break things.

Sorry about the inconvenience here.

This patch has been floating around for a while (albeit not in -next, since
I send my stuff via Joerg) and is heading for 5.3, so you have ages to fix
up your config! It would, of course, be better to configure the IOMMU to
provide mappings for your DMA peripherals, but the trivial config change
will be enough to keep things working. We won't remove that as long as
people are relying on it.

I don't expect most people to run into problems with this change (the new
behaviour matches what SMMUv3 does already).

Will

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Thierry Reding]

[-- Attachment #1.1: Type: text/plain, Size: 5006 bytes --]

On Thu, May 02, 2019 at 12:08:21PM +0100, Will Deacon wrote:
> On Thu, May 02, 2019 at 12:59:12PM +0200, Thierry Reding wrote:
> > On Wed, Apr 24, 2019 at 12:52:31PM +0100, Will Deacon wrote:
> > > On Wed, Apr 24, 2019 at 01:36:58PM +0200, Marc Gonzalez wrote:
> > > > On 04/04/2019 17:00, Will Deacon wrote:
> > > > 
> > > > > On Fri, Mar 01, 2019 at 11:20:17AM -0800, Douglas Anderson wrote:
> > > > >
> > > > >> If you're bisecting why your peripherals stopped working, it's
> > > > >> probably this CL.  Specifically if you see this in your dmesg:
> > > > >>   Unexpected global fault, this could be serious
> > > > >> ...then it's almost certainly this CL.
> > > > >>
> > > > >> Running your IOMMU-enabled peripherals with the IOMMU in bypass mode
> > > > >> is insecure and effectively disables the protection they provide.
> > > > >> There are few reasons to allow unmatched stream bypass, and even fewer
> > > > >> good ones.
> > > > >>
> > > > >> This patch starts the transition over to make it much harder to run
> > > > >> your system insecurely.  Expected steps:
> > > > >>
> > > > >> 1. By default disable bypass (so anyone insecure will notice) but make
> > > > >>    it easy for someone to re-enable bypass with just a KConfig change.
> > > > >>    That's this patch.
> > > > >>
> > > > >> 2. After people have had a little time to come to grips with the fact
> > > > >>    that they need to set their IOMMUs properly and have had time to
> > > > >>    dig into how to do this, the KConfig will be eliminated and bypass
> > > > >>    will simply be disabled.  Folks who are truly upset and still
> > > > >>    haven't fixed their system can either figure out how to add
> > > > >>    'arm-smmu.disable_bypass=n' to their command line or revert the
> > > > >>    patch in their own private kernel.  Of course these folks will be
> > > > >>    less secure.
> > > > >>
> > > > >> Suggested-by: Robin Murphy <robin.murphy@arm.com>
> > > > >> Signed-off-by: Douglas Anderson <dianders@chromium.org>
> > > > >> ---
> > > > >>
> > > > >> Changes in v2:
> > > > >> - Flipped default to 'yes' and changed comments a lot.
> > > > >>
> > > > >>  drivers/iommu/Kconfig    | 25 +++++++++++++++++++++++++
> > > > >>  drivers/iommu/arm-smmu.c |  3 ++-
> > > > >>  2 files changed, 27 insertions(+), 1 deletion(-)
> > > > > 
> > > > > Cheers, I'll pick this one up for 5.2.
> > > > 
> > > > Hello Will,
> > > > 
> > > > You haven't pushed this patch out to linux-next AFAICT.
> > > > 
> > > > Is that expected?
> > > 
> > > It's on my branch for Joerg:
> > > 
> > > https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=for-joerg/arm-smmu/updates
> > > 
> > > which I'll send to him today. My SMMU stuff doesn't go directly into -next.
> > 
> > This made it to linux-next yesterday (less than a week before the merge
> > window opens) and deliberately breaks existing configurations. That's a
> > little rude.
> > 
> > At least give people a fair heads-up and a chance to fix things before
> > you start break things.
> 
> Sorry about the inconvenience here.
> 
> This patch has been floating around for a while (albeit not in -next, since
> I send my stuff via Joerg)

You can't expect people to test random patches from the list if they're
not on Cc. I don't think it's safe to claim that patches have been well
tested until they've been in linux-next for at least a couple of days.

>                            and is heading for 5.3, so you have ages to fix
> up your config!

Last I checked, Joerg applied this for 5.2 because you sent it as part
of your "Updates for 5.2" pull request.

>                 It would, of course, be better to configure the IOMMU to
> provide mappings for your DMA peripherals, but the trivial config change
> will be enough to keep things working. We won't remove that as long as
> people are relying on it.

I don't think the Kconfig option is really useful. People nowadays want
to run standard distribution kernels on their devices, and distribution
maintainers will often rely on kernel developers' guidance on what good
defaults are. This patch suggests that the default should be to disable
bypass, so if this hits 5.2 final and distributions create their kernel
packages, they're likely going to go with this default and potentially
break things for many of their users.

Luckily this seems like it's fairly easy to fix, but given that we're
past v5.1-rc6, fixes for this now need to get special treatment. That
would've been okay if this was a pressing issues, but this is changing
something that's worked this way for ages, so it's hardly urgent.

> I don't expect most people to run into problems with this change (the new
> behaviour matches what SMMUv3 does already).

I see the ARM SMMU v2 used in quite a few DTS files. Not all of these
may be problematic, but I'd be somewhat surprised if Tegra was the only
one impacted.

Thierry

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Will Deacon]

On Thu, May 02, 2019 at 02:45:25PM +0200, Thierry Reding wrote:
> On Thu, May 02, 2019 at 12:08:21PM +0100, Will Deacon wrote:
> > On Thu, May 02, 2019 at 12:59:12PM +0200, Thierry Reding wrote:
> > > This made it to linux-next yesterday (less than a week before the merge
> > > window opens) and deliberately breaks existing configurations. That's a
> > > little rude.
> > > 
> > > At least give people a fair heads-up and a chance to fix things before
> > > you start break things.
> > 
> > Sorry about the inconvenience here.
> > 
> > This patch has been floating around for a while (albeit not in -next, since
> > I send my stuff via Joerg)
> 
> You can't expect people to test random patches from the list if they're
> not on Cc. I don't think it's safe to claim that patches have been well
> tested until they've been in linux-next for at least a couple of days.

I didn't claim it had been well tested. I also don't think being in
linux-next implies a patch has been well tested, to be honest with you.
What I can say is that this has been discussed on the public mailing list
for some time, and the outcome of that discussion is this patch.

> >                            and is heading for 5.3, so you have ages to fix
> > up your config!
> 
> Last I checked, Joerg applied this for 5.2 because you sent it as part
> of your "Updates for 5.2" pull request.

Sorry, I meant 5.2: the kernel that will be released in ~2 months time,
during which you'll be able to fix issues like this one. If we're having
unresolvable issues late in the cycle, then we can clearly revert this
patch.

> >                 It would, of course, be better to configure the IOMMU to
> > provide mappings for your DMA peripherals, but the trivial config change
> > will be enough to keep things working. We won't remove that as long as
> > people are relying on it.
> 
> I don't think the Kconfig option is really useful. People nowadays want
> to run standard distribution kernels on their devices, and distribution
> maintainers will often rely on kernel developers' guidance on what good
> defaults are. This patch suggests that the default should be to disable
> bypass, so if this hits 5.2 final and distributions create their kernel
> packages, they're likely going to go with this default and potentially
> break things for many of their users.

I'd say that people nowadays also want security by default, so it's a tough
crowd to please. You can still fix your .dts (I see there's a patch from Jon
doing that), or pass "arm-smmu.disable_bypass=0" on the cmdline to fix
things if you're not in a position to change the .config for your kernel
image.

> Luckily this seems like it's fairly easy to fix, but given that we're
> past v5.1-rc6, fixes for this now need to get special treatment. That
> would've been okay if this was a pressing issues, but this is changing
> something that's worked this way for ages, so it's hardly urgent.

Why the special treatment? You can continue to merge fixes after the merge
window, no? I feel like I'm missing something in your workflow here.

> > I don't expect most people to run into problems with this change (the new
> > behaviour matches what SMMUv3 does already).
> 
> I see the ARM SMMU v2 used in quite a few DTS files. Not all of these
> may be problematic, but I'd be somewhat surprised if Tegra was the only
> one impacted.

I didn't say this was specific to Tegra, only that I don't expect most
people to see any issue. I'm sure there will be others, and we can fix
them up as they appear. I can't think of a better way of toggling the
default behaviour.

Will

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Robin Murphy]

On 02/05/2019 13:45, Thierry Reding wrote:
> On Thu, May 02, 2019 at 12:08:21PM +0100, Will Deacon wrote:
>> On Thu, May 02, 2019 at 12:59:12PM +0200, Thierry Reding wrote:
>>> On Wed, Apr 24, 2019 at 12:52:31PM +0100, Will Deacon wrote:
>>>> On Wed, Apr 24, 2019 at 01:36:58PM +0200, Marc Gonzalez wrote:
>>>>> On 04/04/2019 17:00, Will Deacon wrote:
>>>>>
>>>>>> On Fri, Mar 01, 2019 at 11:20:17AM -0800, Douglas Anderson wrote:
>>>>>>
>>>>>>> If you're bisecting why your peripherals stopped working, it's
>>>>>>> probably this CL.  Specifically if you see this in your dmesg:
>>>>>>>    Unexpected global fault, this could be serious
>>>>>>> ...then it's almost certainly this CL.
>>>>>>>
>>>>>>> Running your IOMMU-enabled peripherals with the IOMMU in bypass mode
>>>>>>> is insecure and effectively disables the protection they provide.
>>>>>>> There are few reasons to allow unmatched stream bypass, and even fewer
>>>>>>> good ones.
>>>>>>>
>>>>>>> This patch starts the transition over to make it much harder to run
>>>>>>> your system insecurely.  Expected steps:
>>>>>>>
>>>>>>> 1. By default disable bypass (so anyone insecure will notice) but make
>>>>>>>     it easy for someone to re-enable bypass with just a KConfig change.
>>>>>>>     That's this patch.
>>>>>>>
>>>>>>> 2. After people have had a little time to come to grips with the fact
>>>>>>>     that they need to set their IOMMUs properly and have had time to
>>>>>>>     dig into how to do this, the KConfig will be eliminated and bypass
>>>>>>>     will simply be disabled.  Folks who are truly upset and still
>>>>>>>     haven't fixed their system can either figure out how to add
>>>>>>>     'arm-smmu.disable_bypass=n' to their command line or revert the
>>>>>>>     patch in their own private kernel.  Of course these folks will be
>>>>>>>     less secure.
>>>>>>>
>>>>>>> Suggested-by: Robin Murphy <robin.murphy@arm.com>
>>>>>>> Signed-off-by: Douglas Anderson <dianders@chromium.org>
>>>>>>> ---
>>>>>>>
>>>>>>> Changes in v2:
>>>>>>> - Flipped default to 'yes' and changed comments a lot.
>>>>>>>
>>>>>>>   drivers/iommu/Kconfig    | 25 +++++++++++++++++++++++++
>>>>>>>   drivers/iommu/arm-smmu.c |  3 ++-
>>>>>>>   2 files changed, 27 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> Cheers, I'll pick this one up for 5.2.
>>>>>
>>>>> Hello Will,
>>>>>
>>>>> You haven't pushed this patch out to linux-next AFAICT.
>>>>>
>>>>> Is that expected?
>>>>
>>>> It's on my branch for Joerg:
>>>>
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=for-joerg/arm-smmu/updates
>>>>
>>>> which I'll send to him today. My SMMU stuff doesn't go directly into -next.
>>>
>>> This made it to linux-next yesterday (less than a week before the merge
>>> window opens) and deliberately breaks existing configurations. That's a
>>> little rude.
>>>
>>> At least give people a fair heads-up and a chance to fix things before
>>> you start break things.
>>
>> Sorry about the inconvenience here.
>>
>> This patch has been floating around for a while (albeit not in -next, since
>> I send my stuff via Joerg)
> 
> You can't expect people to test random patches from the list if they're
> not on Cc. I don't think it's safe to claim that patches have been well
> tested until they've been in linux-next for at least a couple of days.
> 
>>                             and is heading for 5.3, so you have ages to fix
>> up your config!
> 
> Last I checked, Joerg applied this for 5.2 because you sent it as part
> of your "Updates for 5.2" pull request.
> 
>>                  It would, of course, be better to configure the IOMMU to
>> provide mappings for your DMA peripherals, but the trivial config change
>> will be enough to keep things working. We won't remove that as long as
>> people are relying on it.
> 
> I don't think the Kconfig option is really useful. People nowadays want
> to run standard distribution kernels on their devices, and distribution
> maintainers will often rely on kernel developers' guidance on what good
> defaults are. This patch suggests that the default should be to disable
> bypass, so if this hits 5.2 final and distributions create their kernel
> packages, they're likely going to go with this default and potentially
> break things for many of their users.
> 
> Luckily this seems like it's fairly easy to fix, but given that we're
> past v5.1-rc6, fixes for this now need to get special treatment. That
> would've been okay if this was a pressing issues, but this is changing
> something that's worked this way for ages, so it's hardly urgent.
> 
>> I don't expect most people to run into problems with this change (the new
>> behaviour matches what SMMUv3 does already).
> 
> I see the ARM SMMU v2 used in quite a few DTS files. Not all of these
> may be problematic, but I'd be somewhat surprised if Tegra was the only
> one impacted.

If people have real technical reasons for needing unmatched stream 
bypass, then we'll probably need to consider some sort of per-SMMU 
override anyway, since well-meaning users rendering the system unusable 
by enabling a security option isn't ideal. I'm in the middle of a big 
rework to accommodate platform-specific extensions more easily, so 
that's certainly an angle I can try to plan for.

On the other hand, if it's merely that nobody's yet got round to filling 
out the DT properly, then AIUI that's pretty much exactly what Doug 
wanted to motivate, so that the option *can* be used more widely by 
users who might want it. Hence the provocative title :)

Robin.

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Thierry Reding]

[-- Attachment #1.1: Type: text/plain, Size: 10149 bytes --]

On Thu, May 02, 2019 at 03:27:55PM +0100, Robin Murphy wrote:
> On 02/05/2019 13:45, Thierry Reding wrote:
> > On Thu, May 02, 2019 at 12:08:21PM +0100, Will Deacon wrote:
> > > On Thu, May 02, 2019 at 12:59:12PM +0200, Thierry Reding wrote:
> > > > On Wed, Apr 24, 2019 at 12:52:31PM +0100, Will Deacon wrote:
> > > > > On Wed, Apr 24, 2019 at 01:36:58PM +0200, Marc Gonzalez wrote:
> > > > > > On 04/04/2019 17:00, Will Deacon wrote:
> > > > > > 
> > > > > > > On Fri, Mar 01, 2019 at 11:20:17AM -0800, Douglas Anderson wrote:
> > > > > > > 
> > > > > > > > If you're bisecting why your peripherals stopped working, it's
> > > > > > > > probably this CL.  Specifically if you see this in your dmesg:
> > > > > > > >    Unexpected global fault, this could be serious
> > > > > > > > ...then it's almost certainly this CL.
> > > > > > > > 
> > > > > > > > Running your IOMMU-enabled peripherals with the IOMMU in bypass mode
> > > > > > > > is insecure and effectively disables the protection they provide.
> > > > > > > > There are few reasons to allow unmatched stream bypass, and even fewer
> > > > > > > > good ones.
> > > > > > > > 
> > > > > > > > This patch starts the transition over to make it much harder to run
> > > > > > > > your system insecurely.  Expected steps:
> > > > > > > > 
> > > > > > > > 1. By default disable bypass (so anyone insecure will notice) but make
> > > > > > > >     it easy for someone to re-enable bypass with just a KConfig change.
> > > > > > > >     That's this patch.
> > > > > > > > 
> > > > > > > > 2. After people have had a little time to come to grips with the fact
> > > > > > > >     that they need to set their IOMMUs properly and have had time to
> > > > > > > >     dig into how to do this, the KConfig will be eliminated and bypass
> > > > > > > >     will simply be disabled.  Folks who are truly upset and still
> > > > > > > >     haven't fixed their system can either figure out how to add
> > > > > > > >     'arm-smmu.disable_bypass=n' to their command line or revert the
> > > > > > > >     patch in their own private kernel.  Of course these folks will be
> > > > > > > >     less secure.
> > > > > > > > 
> > > > > > > > Suggested-by: Robin Murphy <robin.murphy@arm.com>
> > > > > > > > Signed-off-by: Douglas Anderson <dianders@chromium.org>
> > > > > > > > ---
> > > > > > > > 
> > > > > > > > Changes in v2:
> > > > > > > > - Flipped default to 'yes' and changed comments a lot.
> > > > > > > > 
> > > > > > > >   drivers/iommu/Kconfig    | 25 +++++++++++++++++++++++++
> > > > > > > >   drivers/iommu/arm-smmu.c |  3 ++-
> > > > > > > >   2 files changed, 27 insertions(+), 1 deletion(-)
> > > > > > > 
> > > > > > > Cheers, I'll pick this one up for 5.2.
> > > > > > 
> > > > > > Hello Will,
> > > > > > 
> > > > > > You haven't pushed this patch out to linux-next AFAICT.
> > > > > > 
> > > > > > Is that expected?
> > > > > 
> > > > > It's on my branch for Joerg:
> > > > > 
> > > > > https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=for-joerg/arm-smmu/updates
> > > > > 
> > > > > which I'll send to him today. My SMMU stuff doesn't go directly into -next.
> > > > 
> > > > This made it to linux-next yesterday (less than a week before the merge
> > > > window opens) and deliberately breaks existing configurations. That's a
> > > > little rude.
> > > > 
> > > > At least give people a fair heads-up and a chance to fix things before
> > > > you start break things.
> > > 
> > > Sorry about the inconvenience here.
> > > 
> > > This patch has been floating around for a while (albeit not in -next, since
> > > I send my stuff via Joerg)
> > 
> > You can't expect people to test random patches from the list if they're
> > not on Cc. I don't think it's safe to claim that patches have been well
> > tested until they've been in linux-next for at least a couple of days.
> > 
> > >                             and is heading for 5.3, so you have ages to fix
> > > up your config!
> > 
> > Last I checked, Joerg applied this for 5.2 because you sent it as part
> > of your "Updates for 5.2" pull request.
> > 
> > >                  It would, of course, be better to configure the IOMMU to
> > > provide mappings for your DMA peripherals, but the trivial config change
> > > will be enough to keep things working. We won't remove that as long as
> > > people are relying on it.
> > 
> > I don't think the Kconfig option is really useful. People nowadays want
> > to run standard distribution kernels on their devices, and distribution
> > maintainers will often rely on kernel developers' guidance on what good
> > defaults are. This patch suggests that the default should be to disable
> > bypass, so if this hits 5.2 final and distributions create their kernel
> > packages, they're likely going to go with this default and potentially
> > break things for many of their users.
> > 
> > Luckily this seems like it's fairly easy to fix, but given that we're
> > past v5.1-rc6, fixes for this now need to get special treatment. That
> > would've been okay if this was a pressing issues, but this is changing
> > something that's worked this way for ages, so it's hardly urgent.
> > 
> > > I don't expect most people to run into problems with this change (the new
> > > behaviour matches what SMMUv3 does already).
> > 
> > I see the ARM SMMU v2 used in quite a few DTS files. Not all of these
> > may be problematic, but I'd be somewhat surprised if Tegra was the only
> > one impacted.
> 
> If people have real technical reasons for needing unmatched stream bypass,
> then we'll probably need to consider some sort of per-SMMU override anyway,
> since well-meaning users rendering the system unusable by enabling a
> security option isn't ideal. I'm in the middle of a big rework to
> accommodate platform-specific extensions more easily, so that's certainly an
> angle I can try to plan for.
> 
> On the other hand, if it's merely that nobody's yet got round to filling out
> the DT properly, then AIUI that's pretty much exactly what Doug wanted to
> motivate, so that the option *can* be used more widely by users who might
> want it. Hence the provocative title :)

I was just rebasing a set of patches that I've been working on for some
time that will eventually allow seamlessly transitioning display from a
firmware-controlled framebuffer to a DRM/KMS framebuffer.

The idea is to mark firmware-allocated buffers as reserved via the
standard reserved-memory DT bindings and hook them up in device tree to
the display controller nodes via memory-region properties. I've got this
working from a firmware/bootloader point of view, but I'm running into a
couple of issues during early boot.

I should mention that as part of this work I've also been meaning to
transition the Tegra SMMU driver that we use on Tegra210 and earlier to
register DMA IOMMU domains and then transition over the display driver
to use that instead of explicit IOMMU usage. One of the issues that I
ran into early on was that the Tegra SMMU would start faulting as soon
as the display controllers had been attached to the DMA IOMMU domain.
This is because the display hardware is still scanning out the frame-
buffer that firmware had set up when the DMA API hooks up to the IOMMU
domain. Currently we don't run into that because we explicitly only
attach to the IOMMU after the display hardware has been reinitialized.

This particular problem was fairly easy to fix by implementing the
IOMMU's ->get_resv_regions() and ->put_resv_regions() callbacks that
parse the memory-region properties for a device and create 1:1 mappings
for each region before "enabling" the domain.

I have patches to implement the same mechanism on Tegra186 where an ARM
SMMU is used. Unfortunately, with this patch to disable bypass by
default, this approach no longer works. The problem is that the ARM SMMU
starts faulting immediately after it's bound to the driver because all
stream IDs will now fault by default (and like I mentioned, the display
controller is still busy scanning out the framebuffer set up by the
firmware). The faults cause a massive amount of interrupt, slowing down
the boot process significantly, and cause the display hardware to read
the framebuffer as all-ones, so the display goes to complete white early
during boot.

Now, I realize that this is precisely what this patch intended. It's
quite nasty for the display hardware to just keep scanning stuff out
from where it may not be allowed to. On the other hand, if we're careful
about marking the firmware framebuffer region reserved, Linux won't be
touching it and it'd be fine for the hardware to keep reading from it.
We do need to make sure to establish that 1:1 mapping, though, to fix
these faults.

So, since you mentioned "real technical reasons", I'm wondering if you
have any ideas how to fix this. I understand that, strictly speaking, we
do violate the Linux boot requirements here (display hardware is not
quiesced), but surely seamless transition from a firmware boot splash to
the kernel display driver is a feature that we want to support.

I suppose an easy way would be to let the firmware add a command-line
argument to enable bypass, but that seems a little coarse and it would
enable bypass in general, again opening up the security issues that you
wanted to close with this.

Perhaps an alternative would be to add a property to the SMMU node that
lists a set of stream IDs for which to enable bypass by default. We
could let the firmware set that when the display hardware has been set
up. That way when the kernel boots we can keep scanning from the
reserved memory and the ARM SMMU driver would not disable bypass for the
display hardware. Only when the display hardware is actually attached to
the IOMMU domain, and the 1:1 mappings have been created would bypass be
disabled, and at that point there should be no SMMU faults anymore, so
we have cleanly transitioned to the kernel.

Any thoughts?

Thierry

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Will Deacon]

On Mon, Aug 19, 2019 at 01:28:56PM +0200, Thierry Reding wrote:
> Perhaps an alternative would be to add a property to the SMMU node that
> lists a set of stream IDs for which to enable bypass by default. We
> could let the firmware set that when the display hardware has been set
> up. That way when the kernel boots we can keep scanning from the
> reserved memory and the ARM SMMU driver would not disable bypass for the
> display hardware. Only when the display hardware is actually attached to
> the IOMMU domain, and the 1:1 mappings have been created would bypass be
> disabled, and at that point there should be no SMMU faults anymore, so
> we have cleanly transitioned to the kernel.
> 
> Any thoughts?

There is currently an extension to IORT under discussion which should
address this problem, so it would make a lot of sense for the DT solution
to follow the same approach. I think it will end up being along the lines
that you suggest, although we won't just enable bypass because that leaves
memory wide open if the device driver doesn't probe and it also creates
an issue because device attach typically happens before the endpoint
driver has probed.

So the flow would look something like:

	- Firmware describes a physical region of memory which must be
	  reserved by the OS.

	- Additionally, firmware describes a master -> reserved memory
	  linkage as part of the IOMMU description.

	- When the IOMMU probes, these reserved memory regions will be
	  mapped 1:1 for the relevant master.

This is similar to RMRR on x86, except that the mappings are intended to
be less rigid and can be torn down if the endpoint driver decides to do
that or for things like device passthrough.

If we get that working, we should update our booting.txt so that DMA is
allowed during boot in the limited cases which this covers.

Will

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Thierry Reding]

[-- Attachment #1.1: Type: text/plain, Size: 2876 bytes --]

On Mon, Aug 19, 2019 at 01:09:18PM +0100, Will Deacon wrote:
> On Mon, Aug 19, 2019 at 01:28:56PM +0200, Thierry Reding wrote:
> > Perhaps an alternative would be to add a property to the SMMU node that
> > lists a set of stream IDs for which to enable bypass by default. We
> > could let the firmware set that when the display hardware has been set
> > up. That way when the kernel boots we can keep scanning from the
> > reserved memory and the ARM SMMU driver would not disable bypass for the
> > display hardware. Only when the display hardware is actually attached to
> > the IOMMU domain, and the 1:1 mappings have been created would bypass be
> > disabled, and at that point there should be no SMMU faults anymore, so
> > we have cleanly transitioned to the kernel.
> > 
> > Any thoughts?
> 
> There is currently an extension to IORT under discussion which should
> address this problem, so it would make a lot of sense for the DT solution
> to follow the same approach. I think it will end up being along the lines
> that you suggest, although we won't just enable bypass because that leaves
> memory wide open if the device driver doesn't probe and it also creates
> an issue because device attach typically happens before the endpoint
> driver has probed.
> 
> So the flow would look something like:
> 
> 	- Firmware describes a physical region of memory which must be
> 	  reserved by the OS.
> 
> 	- Additionally, firmware describes a master -> reserved memory
> 	  linkage as part of the IOMMU description.
> 
> 	- When the IOMMU probes, these reserved memory regions will be
> 	  mapped 1:1 for the relevant master.
> 
> This is similar to RMRR on x86, except that the mappings are intended to
> be less rigid and can be torn down if the endpoint driver decides to do
> that or for things like device passthrough.
> 
> If we get that working, we should update our booting.txt so that DMA is
> allowed during boot in the limited cases which this covers.

Hi Will,

that sounds very interesting. Is this extension being publicly
discussed? If so, do you have any pointers for me to read up on this? As
for device tree, I wonder if perhaps we can achieve this without going
through extra properties. We could, for example, just do a "reverse
lookup" of IOMMU masters by walking the device tree and looking for
nodes that link to an ARM SMMU in their iommus property. Granted, that's
not going to be very efficient, but it would remove the need to
duplicate information in DT. It's also going to be a one-time cost, so
perhaps it would be negligible.

I'm happy to help out with hashing out or implementing something on the
DT side of things. I don't currently have access to any systems with
ACPI, but I've got a bunch of systems that are DT based and that I would
like to see this implemented on.

Thierry

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] iommu/arm-smmu: Break insecure users by disabling bypass by default

[Will Deacon]

On Mon, Aug 19, 2019 at 03:33:27PM +0200, Thierry Reding wrote:
> On Mon, Aug 19, 2019 at 01:09:18PM +0100, Will Deacon wrote:
> > On Mon, Aug 19, 2019 at 01:28:56PM +0200, Thierry Reding wrote:
> > > Perhaps an alternative would be to add a property to the SMMU node that
> > > lists a set of stream IDs for which to enable bypass by default. We
> > > could let the firmware set that when the display hardware has been set
> > > up. That way when the kernel boots we can keep scanning from the
> > > reserved memory and the ARM SMMU driver would not disable bypass for the
> > > display hardware. Only when the display hardware is actually attached to
> > > the IOMMU domain, and the 1:1 mappings have been created would bypass be
> > > disabled, and at that point there should be no SMMU faults anymore, so
> > > we have cleanly transitioned to the kernel.
> > > 
> > > Any thoughts?
> > 
> > There is currently an extension to IORT under discussion which should
> > address this problem, so it would make a lot of sense for the DT solution
> > to follow the same approach. I think it will end up being along the lines
> > that you suggest, although we won't just enable bypass because that leaves
> > memory wide open if the device driver doesn't probe and it also creates
> > an issue because device attach typically happens before the endpoint
> > driver has probed.
> > 
> > So the flow would look something like:
> > 
> > 	- Firmware describes a physical region of memory which must be
> > 	  reserved by the OS.
> > 
> > 	- Additionally, firmware describes a master -> reserved memory
> > 	  linkage as part of the IOMMU description.
> > 
> > 	- When the IOMMU probes, these reserved memory regions will be
> > 	  mapped 1:1 for the relevant master.
> > 
> > This is similar to RMRR on x86, except that the mappings are intended to
> > be less rigid and can be torn down if the endpoint driver decides to do
> > that or for things like device passthrough.
> > 
> > If we get that working, we should update our booting.txt so that DMA is
> > allowed during boot in the limited cases which this covers.
> 
> that sounds very interesting. Is this extension being publicly
> discussed? If so, do you have any pointers for me to read up on this?

Sorry, I don't think it's public :(

> As for device tree, I wonder if perhaps we can achieve this without going
> through extra properties. We could, for example, just do a "reverse
> lookup" of IOMMU masters by walking the device tree and looking for nodes
> that link to an ARM SMMU in their iommus property. Granted, that's not
> going to be very efficient, but it would remove the need to duplicate
> information in DT. It's also going to be a one-time cost, so perhaps it
> would be negligible.

If we can get by with extending the iommu-map entries for the masters,
that would certainly be the neatest imo. Sounds like it's worth a quick
hack, if nothing else.

> I'm happy to help out with hashing out or implementing something on the
> DT side of things. I don't currently have access to any systems with
> ACPI, but I've got a bunch of systems that are DT based and that I would
> like to see this implemented on.

Oh, I'm certainly not saying we should hold the DT changes until the ACPI
side is sorted out, just that we should avoid divergence where we can.
Hopefully my explanation above should be sufficient for that. If you hack
something up that works for DT, I'd be happy to review it.

Will