* [GIT PULL] Intel IOMMU for 4.1
@ 2015-04-26 8:51 David Woodhouse
[not found] ` <1430038266.4833.14.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: David Woodhouse @ 2015-04-26 8:51 UTC (permalink / raw)
To: torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA
[-- Attachment #1.1: Type: text/plain, Size: 2441 bytes --]
Linus,
Please pull the following changes (since v3.19) from
git://git.infradead.org/intel-iommu.git
This lays a little of the groundwork for upcoming Shared Virtual
Memory support — fixing some bogus #defines for capability bits and
adding the new ones, and starting to use the new wider page tables
where we can, in anticipation of actually filling in the new fields
therein.
It also allows graphics devices to be assigned to VM guests again.
This got broken in 3.17 by disallowing assignment of RMRR-afflicted
devices. Like USB, we do understand why there's an RMRR for graphics
devices — and unlike USB, it's actually sane. So we can make an
exception for graphics devices, just as we do USB controllers.
Finally, tone down the warning about the X2APIC_OPT_OUT bit, due to
persistent requests. X2APIC_OPT_OUT was added to the spec as a nasty
hack to allow broken BIOSes to forbid us from using X2APIC when they
do stupid and invasive things and would break if we did.
Someone noticed that since Windows doesn't have full IOMMU support for
DMA protection, setting the X2APIC_OPT_OUT bit made Windows avoid
initialising the IOMMU on the graphics unit altogether.
This means that it would be available for use in "driver mode", where
the IOMMU registers are made available through a BAR of the graphics
device and the graphics driver can do SVM all for itself.
So they started setting the X2APIC_OPT_OUT bit on *all* platforms with
SVM capabilities. And even the platforms which *might*, if the planets
had been aligned correctly, possibly have had SVM capability but which
in practice actually don't.
David Woodhouse (4):
iommu/vt-d: kill bogus ecap_niotlb_iunits()
iommu/vt-d: Allow RMRR on graphics devices too
iommu/vt-d: Add new extended capabilities from v2.3 VT-d specification
iommu/vt-d: support extended root and context entries
Fenghua Yu (1):
iommu/vt-d: Print x2apic opt out info instead of printing a warning
drivers/iommu/intel-iommu.c | 142 +++++++++++++++++-------------------
drivers/iommu/intel_irq_remapping.c | 5 +-
include/linux/intel-iommu.h | 18 ++++-
3 files changed, 82 insertions(+), 83 deletions(-)
--
David Woodhouse Open Source Technology Centre
David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org Intel Corporation
[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <1430038266.4833.14.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2015-04-26 17:33 ` Alex Williamson
[not found] ` <1430069591.8301.91.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: Alex Williamson @ 2015-04-26 17:33 UTC (permalink / raw)
To: David Woodhouse
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
On Sun, 2015-04-26 at 09:51 +0100, David Woodhouse wrote:
> Linus,
>
> Please pull the following changes (since v3.19) from
>
> git://git.infradead.org/intel-iommu.git
>
> This lays a little of the groundwork for upcoming Shared Virtual
> Memory support — fixing some bogus #defines for capability bits and
> adding the new ones, and starting to use the new wider page tables
> where we can, in anticipation of actually filling in the new fields
> therein.
>
> It also allows graphics devices to be assigned to VM guests again.
> This got broken in 3.17 by disallowing assignment of RMRR-afflicted
> devices. Like USB, we do understand why there's an RMRR for graphics
> devices — and unlike USB, it's actually sane. So we can make an
> exception for graphics devices, just as we do USB controllers.
>
> Finally, tone down the warning about the X2APIC_OPT_OUT bit, due to
> persistent requests. X2APIC_OPT_OUT was added to the spec as a nasty
> hack to allow broken BIOSes to forbid us from using X2APIC when they
> do stupid and invasive things and would break if we did.
>
> Someone noticed that since Windows doesn't have full IOMMU support for
> DMA protection, setting the X2APIC_OPT_OUT bit made Windows avoid
> initialising the IOMMU on the graphics unit altogether.
>
> This means that it would be available for use in "driver mode", where
> the IOMMU registers are made available through a BAR of the graphics
> device and the graphics driver can do SVM all for itself.
>
> So they started setting the X2APIC_OPT_OUT bit on *all* platforms with
> SVM capabilities. And even the platforms which *might*, if the planets
> had been aligned correctly, possibly have had SVM capability but which
> in practice actually don't.
>
>
> David Woodhouse (4):
> iommu/vt-d: kill bogus ecap_niotlb_iunits()
> iommu/vt-d: Allow RMRR on graphics devices too
Curious why these weren't posted to the mailing list. This one in
particular not only ignores RMRR on IGD, but any PCI class display
device. We may understand why IGD has RMRRs, but we certainly don't
understand enough to ignore RMRRs for other GPUs, should they exist.
Thanks,
Alex
> iommu/vt-d: Add new extended capabilities from v2.3 VT-d specification
> iommu/vt-d: support extended root and context entries
>
> Fenghua Yu (1):
> iommu/vt-d: Print x2apic opt out info instead of printing a warning
>
> drivers/iommu/intel-iommu.c | 142 +++++++++++++++++-------------------
> drivers/iommu/intel_irq_remapping.c | 5 +-
> include/linux/intel-iommu.h | 18 ++++-
> 3 files changed, 82 insertions(+), 83 deletions(-)
>
>
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <1430069591.8301.91.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
@ 2015-04-26 17:55 ` David Woodhouse
[not found] ` <1430070924.13479.5.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: David Woodhouse @ 2015-04-26 17:55 UTC (permalink / raw)
To: Alex Williamson
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
[-- Attachment #1.1: Type: text/plain, Size: 1296 bytes --]
On Sun, 2015-04-26 at 11:33 -0600, Alex Williamson wrote:
> Curious why these weren't posted to the mailing list.
Apologies for that. I was thinking of this tree as *only* collecting
the stuff for SVM, and had mostly forgotten the bug-fixes. I was going
to post the series once I had it all in better shape... and when it
became apparent that SVM wasn't going to make it for 4.1 I wasn't
actually going to push these early parts that *are* ready at all. But
then I started getting badgered about the X2APIC_OPT_OUT nonsense.
> This one in particular not only ignores RMRR on IGD, but any PCI
> class display device. We may understand why IGD has RMRRs, but we
> certainly don't understand enough to ignore RMRRs for other GPUs,
> should they exist.
If they exist, they'll exist for precisely the same reason — to let
the graphics device continue to render its framebuffer after the IOMMU
is turned on.
For a discrete card with a discrete video BIOS, as opposed to chipset
-integrated stuff, I suspect it's actually much *less* likely that
there's any other weirdness going on.
--
David Woodhouse Open Source Technology Centre
David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org Intel Corporation
[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <1430070924.13479.5.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2015-04-26 18:16 ` Alex Williamson
[not found] ` <1430072207.8301.103.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: Alex Williamson @ 2015-04-26 18:16 UTC (permalink / raw)
To: David Woodhouse
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
On Sun, 2015-04-26 at 18:55 +0100, David Woodhouse wrote:
> On Sun, 2015-04-26 at 11:33 -0600, Alex Williamson wrote:
> > Curious why these weren't posted to the mailing list.
>
> Apologies for that. I was thinking of this tree as *only* collecting
> the stuff for SVM, and had mostly forgotten the bug-fixes. I was going
> to post the series once I had it all in better shape... and when it
> became apparent that SVM wasn't going to make it for 4.1 I wasn't
> actually going to push these early parts that *are* ready at all. But
> then I started getting badgered about the X2APIC_OPT_OUT nonsense.
>
> > This one in particular not only ignores RMRR on IGD, but any PCI
> > class display device. We may understand why IGD has RMRRs, but we
> > certainly don't understand enough to ignore RMRRs for other GPUs,
> > should they exist.
>
> If they exist, they'll exist for precisely the same reason — to let
> the graphics device continue to render its framebuffer after the IOMMU
> is turned on.
>
> For a discrete card with a discrete video BIOS, as opposed to chipset
> -integrated stuff, I suspect it's actually much *less* likely that
> there's any other weirdness going on.
Hmm, we've seen RMRRs used for health monitoring and thermal control on
other devices. These are exactly the kinds of RMRR use cases that
causes us trouble via the IOMMU API. GPUs are potentially the smartest
and most power hungry peripherals in a system. I don't see how thermal
monitoring wouldn't be an attractive target for GPUs. Are you counting
on system vendors not having sufficient pull with the GPU vendors to
hack into their firmware? I had sort of envisioned that we'd need to
match the RMRRs to the use cases we know about for IGD to make sure
nothing sneaks by. We should at least be documenting what the expected
use cases are, the ranges involved, and why we can ignore them. Thanks,
Alex
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <1430072207.8301.103.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
@ 2015-04-27 14:00 ` Linda Knippers
[not found] ` <553E4109.5060903-VXdhtT5mjnY@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: Linda Knippers @ 2015-04-27 14:00 UTC (permalink / raw)
To: Alex Williamson, David Woodhouse
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
On 4/26/2015 2:16 PM, Alex Williamson wrote:
> On Sun, 2015-04-26 at 18:55 +0100, David Woodhouse wrote:
>> On Sun, 2015-04-26 at 11:33 -0600, Alex Williamson wrote:
>>> Curious why these weren't posted to the mailing list.
>>
>> Apologies for that. I was thinking of this tree as *only* collecting
>> the stuff for SVM, and had mostly forgotten the bug-fixes. I was going
>> to post the series once I had it all in better shape... and when it
>> became apparent that SVM wasn't going to make it for 4.1 I wasn't
>> actually going to push these early parts that *are* ready at all. But
>> then I started getting badgered about the X2APIC_OPT_OUT nonsense.
>>
>>> This one in particular not only ignores RMRR on IGD, but any PCI
>>> class display device. We may understand why IGD has RMRRs, but we
>>> certainly don't understand enough to ignore RMRRs for other GPUs,
>>> should they exist.
>>
>> If they exist, they'll exist for precisely the same reason — to let
>> the graphics device continue to render its framebuffer after the IOMMU
>> is turned on.
>>
>> For a discrete card with a discrete video BIOS, as opposed to chipset
>> -integrated stuff, I suspect it's actually much *less* likely that
>> there's any other weirdness going on.
>
> Hmm, we've seen RMRRs used for health monitoring and thermal control on
> other devices. These are exactly the kinds of RMRR use cases that
> causes us trouble via the IOMMU API. GPUs are potentially the smartest
> and most power hungry peripherals in a system. I don't see how thermal
> monitoring wouldn't be an attractive target for GPUs. Are you counting
> on system vendors not having sufficient pull with the GPU vendors to
> hack into their firmware?
I can't speak for other vendors or for other use cases of RMRR but while
HP does use RMRRs for health and thermal monitoring for some devices,
we don't for GPUs.
Older releases of firmware had RMRRs for GPUs only because we had them for
all slots. Firmware since about May 2013 no longer generates them for GPUs.
A platform running old firmware may see an RMRR for a GPU but the memory
region isn't being used. I assume that GPUs have other mechanisms for
reporting health and thermal information to the platform but don't really know.
I mention this because our use of RMRR has gotten attention in the past
so I understand Alex's concern and wanted to clarify our use as it relates
to GPUs. Other vendors may be doing other things so it may still be a
valid concern. This is only a comment on the use case, not on the patch.
> I had sort of envisioned that we'd need to
> match the RMRRs to the use cases we know about for IGD to make sure
> nothing sneaks by. We should at least be documenting what the expected
> use cases are, the ranges involved, and why we can ignore them.
I think that's a good idea.
-- ljk
> Thanks,
>
> Alex
>
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <553E4109.5060903-VXdhtT5mjnY@public.gmane.org>
@ 2015-04-27 15:39 ` Alex Williamson
[not found] ` <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: Alex Williamson @ 2015-04-27 15:39 UTC (permalink / raw)
To: David Woodhouse
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
On Mon, 2015-04-27 at 10:00 -0400, Linda Knippers wrote:
> On 4/26/2015 2:16 PM, Alex Williamson wrote:
> > On Sun, 2015-04-26 at 18:55 +0100, David Woodhouse wrote:
> >> On Sun, 2015-04-26 at 11:33 -0600, Alex Williamson wrote:
> >>> Curious why these weren't posted to the mailing list.
> >>
> >> Apologies for that. I was thinking of this tree as *only* collecting
> >> the stuff for SVM, and had mostly forgotten the bug-fixes. I was going
> >> to post the series once I had it all in better shape... and when it
> >> became apparent that SVM wasn't going to make it for 4.1 I wasn't
> >> actually going to push these early parts that *are* ready at all. But
> >> then I started getting badgered about the X2APIC_OPT_OUT nonsense.
> >>
> >>> This one in particular not only ignores RMRR on IGD, but any PCI
> >>> class display device. We may understand why IGD has RMRRs, but we
> >>> certainly don't understand enough to ignore RMRRs for other GPUs,
> >>> should they exist.
> >>
> >> If they exist, they'll exist for precisely the same reason — to let
> >> the graphics device continue to render its framebuffer after the IOMMU
> >> is turned on.
> >>
> >> For a discrete card with a discrete video BIOS, as opposed to chipset
> >> -integrated stuff, I suspect it's actually much *less* likely that
> >> there's any other weirdness going on.
> >
> > Hmm, we've seen RMRRs used for health monitoring and thermal control on
> > other devices. These are exactly the kinds of RMRR use cases that
> > causes us trouble via the IOMMU API. GPUs are potentially the smartest
> > and most power hungry peripherals in a system. I don't see how thermal
> > monitoring wouldn't be an attractive target for GPUs. Are you counting
> > on system vendors not having sufficient pull with the GPU vendors to
> > hack into their firmware?
>
> I can't speak for other vendors or for other use cases of RMRR but while
> HP does use RMRRs for health and thermal monitoring for some devices,
> we don't for GPUs.
>
> Older releases of firmware had RMRRs for GPUs only because we had them for
> all slots. Firmware since about May 2013 no longer generates them for GPUs.
> A platform running old firmware may see an RMRR for a GPU but the memory
> region isn't being used. I assume that GPUs have other mechanisms for
> reporting health and thermal information to the platform but don't really know.
>
> I mention this because our use of RMRR has gotten attention in the past
> so I understand Alex's concern and wanted to clarify our use as it relates
> to GPUs. Other vendors may be doing other things so it may still be a
> valid concern. This is only a comment on the use case, not on the patch.
>
> > I had sort of envisioned that we'd need to
> > match the RMRRs to the use cases we know about for IGD to make sure
> > nothing sneaks by. We should at least be documenting what the expected
> > use cases are, the ranges involved, and why we can ignore them.
>
> I think that's a good idea.
David,
Since this got pulled anyway, do you plan to follow-up with patches to
limit the graphics RMRR exception to the known and acceptable uses and
document them, or should I send a revert patch? I don't think what we
have here is acceptable going forward or being backported to stable.
Thanks,
Alex
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
@ 2015-04-27 15:48 ` Linus Torvalds
[not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-27 17:12 ` David Woodhouse
1 sibling, 1 reply; 12+ messages in thread
From: Linus Torvalds @ 2015-04-27 15:48 UTC (permalink / raw)
To: Alex Williamson
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
David Woodhouse
On Mon, Apr 27, 2015 at 8:39 AM, Alex Williamson
<alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>
> Since this got pulled anyway, do you plan to follow-up with patches to
> limit the graphics RMRR exception to the known and acceptable uses and
> document them, or should I send a revert patch? I don't think what we
> have here is acceptable going forward or being backported to stable.
> Thanks,
Does anybody actually *care* about RMRR?
The thing is shit. We know it's shit. We already had Linda pipe up to
tell us that there are bogus RMRR's for ever slot in older machines.
And we already ignore RMRR for USB controllers because it was just
useless garbage.
Is there any reason to *not* just ignore RMRR for all video devices
like we do now? Seriously?
I'm *so* not impressed with firmware tables. These things are always
uniformly wrong. We should strive to generate the information from our
actual hardware knowledge, with BIOS tables being the absolutely least
trusted source of information.
Linus
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2015-04-27 16:09 ` Linda Knippers
2015-04-27 16:16 ` Alex Williamson
2015-04-27 17:28 ` David Woodhouse
2 siblings, 0 replies; 12+ messages in thread
From: Linda Knippers @ 2015-04-27 16:09 UTC (permalink / raw)
To: Linus Torvalds, Alex Williamson
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
David Woodhouse
On 4/27/2015 11:48 AM, Linus Torvalds wrote:
> On Mon, Apr 27, 2015 at 8:39 AM, Alex Williamson
> <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>>
>> Since this got pulled anyway, do you plan to follow-up with patches to
>> limit the graphics RMRR exception to the known and acceptable uses and
>> document them, or should I send a revert patch? I don't think what we
>> have here is acceptable going forward or being backported to stable.
>> Thanks,
>
> Does anybody actually *care* about RMRR?
I care, just not so much for GPUs.
> The thing is shit. We know it's shit. We already had Linda pipe up to
> tell us that there are bogus RMRR's for ever slot in older machines.
I'd call them unnecessary.
> And we already ignore RMRR for USB controllers because it was just
> useless garbage.
>
> Is there any reason to *not* just ignore RMRR for all video devices
> like we do now? Seriously?
I was only speaking about one particular use case of RMRR for video
devices. I'm not sure what anyone else is doing with them.
> I'm *so* not impressed with firmware tables. These things are always
> uniformly wrong. We should strive to generate the information from our
> actual hardware knowledge, with BIOS tables being the absolutely least
> trusted source of information.
In some cases though, you really do need the firmware to provide the tables.
Too few RMRRs are worse than too many.
-- ljk
>
> Linus
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-27 16:09 ` Linda Knippers
@ 2015-04-27 16:16 ` Alex Williamson
2015-04-27 17:28 ` David Woodhouse
2 siblings, 0 replies; 12+ messages in thread
From: Alex Williamson @ 2015-04-27 16:16 UTC (permalink / raw)
To: Linus Torvalds
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
David Woodhouse
On Mon, 2015-04-27 at 08:48 -0700, Linus Torvalds wrote:
> On Mon, Apr 27, 2015 at 8:39 AM, Alex Williamson
> <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> >
> > Since this got pulled anyway, do you plan to follow-up with patches to
> > limit the graphics RMRR exception to the known and acceptable uses and
> > document them, or should I send a revert patch? I don't think what we
> > have here is acceptable going forward or being backported to stable.
> > Thanks,
>
> Does anybody actually *care* about RMRR?
>
> The thing is shit. We know it's shit. We already had Linda pipe up to
> tell us that there are bogus RMRR's for ever slot in older machines.
> And we already ignore RMRR for USB controllers because it was just
> useless garbage.
>
> Is there any reason to *not* just ignore RMRR for all video devices
> like we do now? Seriously?
>
> I'm *so* not impressed with firmware tables. These things are always
> uniformly wrong. We should strive to generate the information from our
> actual hardware knowledge, with BIOS tables being the absolutely least
> trusted source of information.
It's true that there are RMRRs that are generated simply as placeholders
on some systems and not actually used. However, it's also the case that
some RMRRs are actively used and used in such a way that if we ignore
them, we end up with corrupted VM memory because the data reporting done
through the RMRR continues whether the IOVA is mapped to the intended
reserved host address or memory backing the VM. From the kernel
perspective, I don't see how we can tell the difference between
placeholders and active RMRRs, nor do we really have any opportunity to
introspect that this is occurring. In this data reporting mode, RMRRs
are just a mechanism the platform can use to configure side-band data
reporting between devices, outside of the scope or control of then
kernel.
I certainly don't know of any display devices that do this kind of data
reporting, but I'm not thrilled about assuming they don't, or won't
exist. Thanks,
Alex
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-27 15:48 ` Linus Torvalds
@ 2015-04-27 17:12 ` David Woodhouse
[not found] ` <1430154752.6377.19.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
1 sibling, 1 reply; 12+ messages in thread
From: David Woodhouse @ 2015-04-27 17:12 UTC (permalink / raw)
To: Alex Williamson
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
[-- Attachment #1.1: Type: text/plain, Size: 1303 bytes --]
On Mon, 2015-04-27 at 09:39 -0600, Alex Williamson wrote:
> David,
>
> Since this got pulled anyway, do you plan to follow-up with patches to
> limit the graphics RMRR exception to the known and acceptable uses and
> document them, or should I send a revert patch? I don't think what we
> have here is acceptable going forward or being backported to stable.
> Thanks,
I didn't plan to, no. So far we *only* know of acceptable uses of RMRR
on graphics devices, and the false positives that Linda mentioned.
The only cases we ever saw of RMRR being *problematic* when we tried
to assign devices to guests were non-graphics devices, weren't they?
Although obviously I'm looking into the oops that sl4ever reported;
currently I'm wondering if that was broken even before your patch that
blacklisted all RMRR-afflicted devices. It looks like
device_to_iommu() doesn't return the correct results for devices
behind a disabled IOMMU; it incorrectly attributes them to a catch-all
IOMMU instead.
Or perhaps this is probably tied into the fact that your patch broke
iommu=pt for RMRR-afflicted devices too. They currently get full
translation. Which surely shouldn't be necessary, since passthrough
mode would obviously *also* preserve the RMRR regions of 1:1 mappings.
--
dwmw2
[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-27 16:09 ` Linda Knippers
2015-04-27 16:16 ` Alex Williamson
@ 2015-04-27 17:28 ` David Woodhouse
2 siblings, 0 replies; 12+ messages in thread
From: David Woodhouse @ 2015-04-27 17:28 UTC (permalink / raw)
To: Linus Torvalds; +Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA
[-- Attachment #1.1: Type: text/plain, Size: 3948 bytes --]
On Mon, 2015-04-27 at 08:48 -0700, Linus Torvalds wrote:
> On Mon, Apr 27, 2015 at 8:39 AM, Alex Williamson
> <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> >
> > Since this got pulled anyway, do you plan to follow-up with
> > patches to
> > limit the graphics RMRR exception to the known and acceptable uses
> > and
> > document them, or should I send a revert patch? I don't think
> > what we
> > have here is acceptable going forward or being backported to
> > stable.
> > Thanks,
>
> Does anybody actually *care* about RMRR?
>
> The thing is shit. We know it's shit. We already had Linda pipe up to
> tell us that there are bogus RMRR's for ever slot in older machines.
> And we already ignore RMRR for USB controllers because it was just
> useless garbage.
RMRR for USB controllers was useless garbage because any operating
system new enough to enable the IOMMU is *also* going to have USB
support, so shouldn't need to rely on the BIOS doing legacy
keyboard/mouse emulation. Which is the only reason the firmware would
need to keep doing DMA from the USB controllers after we boot.
They could have just said "quiesce the USB BIOS stuff before turning
on the IOMMU" and they wouldn't have needed RMRR for USB.
But even so, we don't just "ignore" the RMRR. We do set it the 1:1
mapping as requested, otherwise we'd get a stream of faults and the
BIOS might even lock up in SMM code if its error handling is crap.
The thing we *don't* do is faithfully preserve that 1:1 mapping for
ever. If we assign the device to a VM guest, the RMRR mapping is lost.
We've never preserved those mappings, and it's only ever been a
problem on the insane crap that HP does with firmware-controlled DMA
from stuff like storage controllers, which persists even at runtime
after we're *using* said controller.
(You might think it would have been easy enough for them to do their
special DMA from another B-D-F but perhaps that would have been too
easy for us.)
It's extremely hard to preserve those mappings when assigning to a
guest, because we'd actually have to poke a corresponding hole in the
guest's physical address space (because in this case GPA==IOVA and
those IOVAs are being used by the BIOS with a 1:1 mapping to HPA).
Hence Alex's patch which blacklisted the 'unknown' RMRR-afflicted
devices and prevented them from being assigned to guests (and also it
seems from honouring 'iommu=pt'; qv.).
> Is there any reason to *not* just ignore RMRR for all video devices
> like we do now? Seriously?
The video case is actually the *sanest* use of RMRR. With a
framebuffer in main memory, the RMRR is the only thing that lets the
graphics device continue to render pixels from the framebuffer after
the IOMMU is initialised. So we *definitely* can't just 'ignore' the
RMRR there. And we don't.
But assignment of such graphics devices to guests *has* allegedly been
working (and iommu=pt is kind of nice too because of the performance
issues), and there's no reason for them to be excluded. They can be
lumped in with the USB devices as "OK to assign to guests". We lose
the 1:1 mapping indicated by the RMRR but that's OK because the guest
has to initialise the hardware entirely for itself and set up its
*own* framebuffer.
> I'm *so* not impressed with firmware tables. These things are always
> uniformly wrong. We should strive to generate the information from
> our actual hardware knowledge, with BIOS tables being the absolutely
> least trusted source of information.
... which is why the whole of the VT-d design, exposing this crap
through firmware tables instead of hardware discoverability, makes me
cringe. But while I am tilting at windmills internally to at least get
people to realise why that's such a design flaw, we still have to deal
with this crap. And with the insane things that HP abuse it for.
--
dwmw2
[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1
[not found] ` <1430154752.6377.19.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2015-04-27 17:30 ` Alex Williamson
0 siblings, 0 replies; 12+ messages in thread
From: Alex Williamson @ 2015-04-27 17:30 UTC (permalink / raw)
To: David Woodhouse
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
On Mon, 2015-04-27 at 18:12 +0100, David Woodhouse wrote:
> On Mon, 2015-04-27 at 09:39 -0600, Alex Williamson wrote:
> > David,
> >
> > Since this got pulled anyway, do you plan to follow-up with patches to
> > limit the graphics RMRR exception to the known and acceptable uses and
> > document them, or should I send a revert patch? I don't think what we
> > have here is acceptable going forward or being backported to stable.
> > Thanks,
>
> I didn't plan to, no. So far we *only* know of acceptable uses of RMRR
> on graphics devices, and the false positives that Linda mentioned.
>
> The only cases we ever saw of RMRR being *problematic* when we tried
> to assign devices to guests were non-graphics devices, weren't they?
That's true, but if we're only trying to enable IGD assignment (which of
course doesn't work yet anyway), why add the entire PCI class rather
than perhaps at least testing for vendor ID 8086 and pci_is_root_bus()?
I don't see why we want to open the door any wider than necessary.
> Although obviously I'm looking into the oops that sl4ever reported;
> currently I'm wondering if that was broken even before your patch that
> blacklisted all RMRR-afflicted devices. It looks like
> device_to_iommu() doesn't return the correct results for devices
> behind a disabled IOMMU; it incorrectly attributes them to a catch-all
> IOMMU instead.
>
> Or perhaps this is probably tied into the fact that your patch broke
> iommu=pt for RMRR-afflicted devices too. They currently get full
> translation. Which surely shouldn't be necessary, since passthrough
> mode would obviously *also* preserve the RMRR regions of 1:1 mappings.
I think you're actually referring to:
commit ea2447f700cab264019b52e2b417d689e052dcfd
Author: Tom Mingarelli <thomas.mingarelli-VXdhtT5mjnY@public.gmane.org>
Date: Tue Nov 20 19:43:17 2012 +0000
intel-iommu: Prevent devices with RMRRs from being placed into SI Domain
This patch is to prevent non-USB devices that have RMRRs associated with the
being placed into the SI Domain during init. This fixes the issue where the
for devices being placed in and out of the SI Domain gets lost.
Signed-off-by: Thomas Mingarelli <thomas.mingarelli-VXdhtT5mjnY@public.gmane.org>
Tested-by: Shuah Khan <shuah.khan-VXdhtT5mjnY@public.gmane.org>
Reviewed-by: Donald Dutile <ddutile-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Reviewed-by: Alex Williamson <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
That patch has been present since v3.8 and was only cosmetically
affected by my patch with a shared helper function. Thanks,
Alex
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2015-04-27 17:30 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-04-26 8:51 [GIT PULL] Intel IOMMU for 4.1 David Woodhouse
[not found] ` <1430038266.4833.14.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-04-26 17:33 ` Alex Williamson
[not found] ` <1430069591.8301.91.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-26 17:55 ` David Woodhouse
[not found] ` <1430070924.13479.5.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-04-26 18:16 ` Alex Williamson
[not found] ` <1430072207.8301.103.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-27 14:00 ` Linda Knippers
[not found] ` <553E4109.5060903-VXdhtT5mjnY@public.gmane.org>
2015-04-27 15:39 ` Alex Williamson
[not found] ` <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-27 15:48 ` Linus Torvalds
[not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-27 16:09 ` Linda Knippers
2015-04-27 16:16 ` Alex Williamson
2015-04-27 17:28 ` David Woodhouse
2015-04-27 17:12 ` David Woodhouse
[not found] ` <1430154752.6377.19.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-04-27 17:30 ` Alex Williamson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox