* [GIT PULL] Intel IOMMU for 4.1
@ 2015-04-26 8:51 David Woodhouse
[not found] ` <1430038266.4833.14.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: David Woodhouse @ 2015-04-26 8:51 UTC (permalink / raw)
To: torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA
[-- Attachment #1.1: Type: text/plain, Size: 2441 bytes --]
Linus,
Please pull the following changes (since v3.19) from
git://git.infradead.org/intel-iommu.git
This lays a little of the groundwork for upcoming Shared Virtual
Memory support — fixing some bogus #defines for capability bits and
adding the new ones, and starting to use the new wider page tables
where we can, in anticipation of actually filling in the new fields
therein.
It also allows graphics devices to be assigned to VM guests again.
This got broken in 3.17 by disallowing assignment of RMRR-afflicted
devices. Like USB, we do understand why there's an RMRR for graphics
devices — and unlike USB, it's actually sane. So we can make an
exception for graphics devices, just as we do USB controllers.
Finally, tone down the warning about the X2APIC_OPT_OUT bit, due to
persistent requests. X2APIC_OPT_OUT was added to the spec as a nasty
hack to allow broken BIOSes to forbid us from using X2APIC when they
do stupid and invasive things and would break if we did.
Someone noticed that since Windows doesn't have full IOMMU support for
DMA protection, setting the X2APIC_OPT_OUT bit made Windows avoid
initialising the IOMMU on the graphics unit altogether.
This means that it would be available for use in "driver mode", where
the IOMMU registers are made available through a BAR of the graphics
device and the graphics driver can do SVM all for itself.
So they started setting the X2APIC_OPT_OUT bit on *all* platforms with
SVM capabilities. And even the platforms which *might*, if the planets
had been aligned correctly, possibly have had SVM capability but which
in practice actually don't.
David Woodhouse (4):
iommu/vt-d: kill bogus ecap_niotlb_iunits()
iommu/vt-d: Allow RMRR on graphics devices too
iommu/vt-d: Add new extended capabilities from v2.3 VT-d specification
iommu/vt-d: support extended root and context entries
Fenghua Yu (1):
iommu/vt-d: Print x2apic opt out info instead of printing a warning
drivers/iommu/intel-iommu.c | 142 +++++++++++++++++-------------------
drivers/iommu/intel_irq_remapping.c | 5 +-
include/linux/intel-iommu.h | 18 ++++-
3 files changed, 82 insertions(+), 83 deletions(-)
--
David Woodhouse Open Source Technology Centre
David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org Intel Corporation
[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread[parent not found: <1430038266.4833.14.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>]
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <1430038266.4833.14.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> @ 2015-04-26 17:33 ` Alex Williamson [not found] ` <1430069591.8301.91.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 12+ messages in thread From: Alex Williamson @ 2015-04-26 17:33 UTC (permalink / raw) To: David Woodhouse Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b On Sun, 2015-04-26 at 09:51 +0100, David Woodhouse wrote: > Linus, > > Please pull the following changes (since v3.19) from > > git://git.infradead.org/intel-iommu.git > > This lays a little of the groundwork for upcoming Shared Virtual > Memory support — fixing some bogus #defines for capability bits and > adding the new ones, and starting to use the new wider page tables > where we can, in anticipation of actually filling in the new fields > therein. > > It also allows graphics devices to be assigned to VM guests again. > This got broken in 3.17 by disallowing assignment of RMRR-afflicted > devices. Like USB, we do understand why there's an RMRR for graphics > devices — and unlike USB, it's actually sane. So we can make an > exception for graphics devices, just as we do USB controllers. > > Finally, tone down the warning about the X2APIC_OPT_OUT bit, due to > persistent requests. X2APIC_OPT_OUT was added to the spec as a nasty > hack to allow broken BIOSes to forbid us from using X2APIC when they > do stupid and invasive things and would break if we did. > > Someone noticed that since Windows doesn't have full IOMMU support for > DMA protection, setting the X2APIC_OPT_OUT bit made Windows avoid > initialising the IOMMU on the graphics unit altogether. > > This means that it would be available for use in "driver mode", where > the IOMMU registers are made available through a BAR of the graphics > device and the graphics driver can do SVM all for itself. > > So they started setting the X2APIC_OPT_OUT bit on *all* platforms with > SVM capabilities. And even the platforms which *might*, if the planets > had been aligned correctly, possibly have had SVM capability but which > in practice actually don't. > > > David Woodhouse (4): > iommu/vt-d: kill bogus ecap_niotlb_iunits() > iommu/vt-d: Allow RMRR on graphics devices too Curious why these weren't posted to the mailing list. This one in particular not only ignores RMRR on IGD, but any PCI class display device. We may understand why IGD has RMRRs, but we certainly don't understand enough to ignore RMRRs for other GPUs, should they exist. Thanks, Alex > iommu/vt-d: Add new extended capabilities from v2.3 VT-d specification > iommu/vt-d: support extended root and context entries > > Fenghua Yu (1): > iommu/vt-d: Print x2apic opt out info instead of printing a warning > > drivers/iommu/intel-iommu.c | 142 +++++++++++++++++------------------- > drivers/iommu/intel_irq_remapping.c | 5 +- > include/linux/intel-iommu.h | 18 ++++- > 3 files changed, 82 insertions(+), 83 deletions(-) > > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <1430069591.8301.91.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <1430069591.8301.91.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2015-04-26 17:55 ` David Woodhouse [not found] ` <1430070924.13479.5.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> 0 siblings, 1 reply; 12+ messages in thread From: David Woodhouse @ 2015-04-26 17:55 UTC (permalink / raw) To: Alex Williamson Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b [-- Attachment #1.1: Type: text/plain, Size: 1296 bytes --] On Sun, 2015-04-26 at 11:33 -0600, Alex Williamson wrote: > Curious why these weren't posted to the mailing list. Apologies for that. I was thinking of this tree as *only* collecting the stuff for SVM, and had mostly forgotten the bug-fixes. I was going to post the series once I had it all in better shape... and when it became apparent that SVM wasn't going to make it for 4.1 I wasn't actually going to push these early parts that *are* ready at all. But then I started getting badgered about the X2APIC_OPT_OUT nonsense. > This one in particular not only ignores RMRR on IGD, but any PCI > class display device. We may understand why IGD has RMRRs, but we > certainly don't understand enough to ignore RMRRs for other GPUs, > should they exist. If they exist, they'll exist for precisely the same reason — to let the graphics device continue to render its framebuffer after the IOMMU is turned on. For a discrete card with a discrete video BIOS, as opposed to chipset -integrated stuff, I suspect it's actually much *less* likely that there's any other weirdness going on. -- David Woodhouse Open Source Technology Centre David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org Intel Corporation [-- Attachment #1.2: smime.p7s --] [-- Type: application/x-pkcs7-signature, Size: 5745 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <1430070924.13479.5.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>]
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <1430070924.13479.5.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> @ 2015-04-26 18:16 ` Alex Williamson [not found] ` <1430072207.8301.103.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 12+ messages in thread From: Alex Williamson @ 2015-04-26 18:16 UTC (permalink / raw) To: David Woodhouse Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b On Sun, 2015-04-26 at 18:55 +0100, David Woodhouse wrote: > On Sun, 2015-04-26 at 11:33 -0600, Alex Williamson wrote: > > Curious why these weren't posted to the mailing list. > > Apologies for that. I was thinking of this tree as *only* collecting > the stuff for SVM, and had mostly forgotten the bug-fixes. I was going > to post the series once I had it all in better shape... and when it > became apparent that SVM wasn't going to make it for 4.1 I wasn't > actually going to push these early parts that *are* ready at all. But > then I started getting badgered about the X2APIC_OPT_OUT nonsense. > > > This one in particular not only ignores RMRR on IGD, but any PCI > > class display device. We may understand why IGD has RMRRs, but we > > certainly don't understand enough to ignore RMRRs for other GPUs, > > should they exist. > > If they exist, they'll exist for precisely the same reason — to let > the graphics device continue to render its framebuffer after the IOMMU > is turned on. > > For a discrete card with a discrete video BIOS, as opposed to chipset > -integrated stuff, I suspect it's actually much *less* likely that > there's any other weirdness going on. Hmm, we've seen RMRRs used for health monitoring and thermal control on other devices. These are exactly the kinds of RMRR use cases that causes us trouble via the IOMMU API. GPUs are potentially the smartest and most power hungry peripherals in a system. I don't see how thermal monitoring wouldn't be an attractive target for GPUs. Are you counting on system vendors not having sufficient pull with the GPU vendors to hack into their firmware? I had sort of envisioned that we'd need to match the RMRRs to the use cases we know about for IGD to make sure nothing sneaks by. We should at least be documenting what the expected use cases are, the ranges involved, and why we can ignore them. Thanks, Alex _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <1430072207.8301.103.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <1430072207.8301.103.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2015-04-27 14:00 ` Linda Knippers [not found] ` <553E4109.5060903-VXdhtT5mjnY@public.gmane.org> 0 siblings, 1 reply; 12+ messages in thread From: Linda Knippers @ 2015-04-27 14:00 UTC (permalink / raw) To: Alex Williamson, David Woodhouse Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b On 4/26/2015 2:16 PM, Alex Williamson wrote: > On Sun, 2015-04-26 at 18:55 +0100, David Woodhouse wrote: >> On Sun, 2015-04-26 at 11:33 -0600, Alex Williamson wrote: >>> Curious why these weren't posted to the mailing list. >> >> Apologies for that. I was thinking of this tree as *only* collecting >> the stuff for SVM, and had mostly forgotten the bug-fixes. I was going >> to post the series once I had it all in better shape... and when it >> became apparent that SVM wasn't going to make it for 4.1 I wasn't >> actually going to push these early parts that *are* ready at all. But >> then I started getting badgered about the X2APIC_OPT_OUT nonsense. >> >>> This one in particular not only ignores RMRR on IGD, but any PCI >>> class display device. We may understand why IGD has RMRRs, but we >>> certainly don't understand enough to ignore RMRRs for other GPUs, >>> should they exist. >> >> If they exist, they'll exist for precisely the same reason — to let >> the graphics device continue to render its framebuffer after the IOMMU >> is turned on. >> >> For a discrete card with a discrete video BIOS, as opposed to chipset >> -integrated stuff, I suspect it's actually much *less* likely that >> there's any other weirdness going on. > > Hmm, we've seen RMRRs used for health monitoring and thermal control on > other devices. These are exactly the kinds of RMRR use cases that > causes us trouble via the IOMMU API. GPUs are potentially the smartest > and most power hungry peripherals in a system. I don't see how thermal > monitoring wouldn't be an attractive target for GPUs. Are you counting > on system vendors not having sufficient pull with the GPU vendors to > hack into their firmware? I can't speak for other vendors or for other use cases of RMRR but while HP does use RMRRs for health and thermal monitoring for some devices, we don't for GPUs. Older releases of firmware had RMRRs for GPUs only because we had them for all slots. Firmware since about May 2013 no longer generates them for GPUs. A platform running old firmware may see an RMRR for a GPU but the memory region isn't being used. I assume that GPUs have other mechanisms for reporting health and thermal information to the platform but don't really know. I mention this because our use of RMRR has gotten attention in the past so I understand Alex's concern and wanted to clarify our use as it relates to GPUs. Other vendors may be doing other things so it may still be a valid concern. This is only a comment on the use case, not on the patch. > I had sort of envisioned that we'd need to > match the RMRRs to the use cases we know about for IGD to make sure > nothing sneaks by. We should at least be documenting what the expected > use cases are, the ranges involved, and why we can ignore them. I think that's a good idea. -- ljk > Thanks, > > Alex > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <553E4109.5060903-VXdhtT5mjnY@public.gmane.org>]
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <553E4109.5060903-VXdhtT5mjnY@public.gmane.org> @ 2015-04-27 15:39 ` Alex Williamson [not found] ` <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 12+ messages in thread From: Alex Williamson @ 2015-04-27 15:39 UTC (permalink / raw) To: David Woodhouse Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b On Mon, 2015-04-27 at 10:00 -0400, Linda Knippers wrote: > On 4/26/2015 2:16 PM, Alex Williamson wrote: > > On Sun, 2015-04-26 at 18:55 +0100, David Woodhouse wrote: > >> On Sun, 2015-04-26 at 11:33 -0600, Alex Williamson wrote: > >>> Curious why these weren't posted to the mailing list. > >> > >> Apologies for that. I was thinking of this tree as *only* collecting > >> the stuff for SVM, and had mostly forgotten the bug-fixes. I was going > >> to post the series once I had it all in better shape... and when it > >> became apparent that SVM wasn't going to make it for 4.1 I wasn't > >> actually going to push these early parts that *are* ready at all. But > >> then I started getting badgered about the X2APIC_OPT_OUT nonsense. > >> > >>> This one in particular not only ignores RMRR on IGD, but any PCI > >>> class display device. We may understand why IGD has RMRRs, but we > >>> certainly don't understand enough to ignore RMRRs for other GPUs, > >>> should they exist. > >> > >> If they exist, they'll exist for precisely the same reason — to let > >> the graphics device continue to render its framebuffer after the IOMMU > >> is turned on. > >> > >> For a discrete card with a discrete video BIOS, as opposed to chipset > >> -integrated stuff, I suspect it's actually much *less* likely that > >> there's any other weirdness going on. > > > > Hmm, we've seen RMRRs used for health monitoring and thermal control on > > other devices. These are exactly the kinds of RMRR use cases that > > causes us trouble via the IOMMU API. GPUs are potentially the smartest > > and most power hungry peripherals in a system. I don't see how thermal > > monitoring wouldn't be an attractive target for GPUs. Are you counting > > on system vendors not having sufficient pull with the GPU vendors to > > hack into their firmware? > > I can't speak for other vendors or for other use cases of RMRR but while > HP does use RMRRs for health and thermal monitoring for some devices, > we don't for GPUs. > > Older releases of firmware had RMRRs for GPUs only because we had them for > all slots. Firmware since about May 2013 no longer generates them for GPUs. > A platform running old firmware may see an RMRR for a GPU but the memory > region isn't being used. I assume that GPUs have other mechanisms for > reporting health and thermal information to the platform but don't really know. > > I mention this because our use of RMRR has gotten attention in the past > so I understand Alex's concern and wanted to clarify our use as it relates > to GPUs. Other vendors may be doing other things so it may still be a > valid concern. This is only a comment on the use case, not on the patch. > > > I had sort of envisioned that we'd need to > > match the RMRRs to the use cases we know about for IGD to make sure > > nothing sneaks by. We should at least be documenting what the expected > > use cases are, the ranges involved, and why we can ignore them. > > I think that's a good idea. David, Since this got pulled anyway, do you plan to follow-up with patches to limit the graphics RMRR exception to the known and acceptable uses and document them, or should I send a revert patch? I don't think what we have here is acceptable going forward or being backported to stable. Thanks, Alex _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2015-04-27 15:48 ` Linus Torvalds [not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2015-04-27 17:12 ` David Woodhouse 1 sibling, 1 reply; 12+ messages in thread From: Linus Torvalds @ 2015-04-27 15:48 UTC (permalink / raw) To: Alex Williamson Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, David Woodhouse On Mon, Apr 27, 2015 at 8:39 AM, Alex Williamson <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote: > > Since this got pulled anyway, do you plan to follow-up with patches to > limit the graphics RMRR exception to the known and acceptable uses and > document them, or should I send a revert patch? I don't think what we > have here is acceptable going forward or being backported to stable. > Thanks, Does anybody actually *care* about RMRR? The thing is shit. We know it's shit. We already had Linda pipe up to tell us that there are bogus RMRR's for ever slot in older machines. And we already ignore RMRR for USB controllers because it was just useless garbage. Is there any reason to *not* just ignore RMRR for all video devices like we do now? Seriously? I'm *so* not impressed with firmware tables. These things are always uniformly wrong. We should strive to generate the information from our actual hardware knowledge, with BIOS tables being the absolutely least trusted source of information. Linus ^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2015-04-27 16:09 ` Linda Knippers 2015-04-27 16:16 ` Alex Williamson 2015-04-27 17:28 ` David Woodhouse 2 siblings, 0 replies; 12+ messages in thread From: Linda Knippers @ 2015-04-27 16:09 UTC (permalink / raw) To: Linus Torvalds, Alex Williamson Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, David Woodhouse On 4/27/2015 11:48 AM, Linus Torvalds wrote: > On Mon, Apr 27, 2015 at 8:39 AM, Alex Williamson > <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote: >> >> Since this got pulled anyway, do you plan to follow-up with patches to >> limit the graphics RMRR exception to the known and acceptable uses and >> document them, or should I send a revert patch? I don't think what we >> have here is acceptable going forward or being backported to stable. >> Thanks, > > Does anybody actually *care* about RMRR? I care, just not so much for GPUs. > The thing is shit. We know it's shit. We already had Linda pipe up to > tell us that there are bogus RMRR's for ever slot in older machines. I'd call them unnecessary. > And we already ignore RMRR for USB controllers because it was just > useless garbage. > > Is there any reason to *not* just ignore RMRR for all video devices > like we do now? Seriously? I was only speaking about one particular use case of RMRR for video devices. I'm not sure what anyone else is doing with them. > I'm *so* not impressed with firmware tables. These things are always > uniformly wrong. We should strive to generate the information from our > actual hardware knowledge, with BIOS tables being the absolutely least > trusted source of information. In some cases though, you really do need the firmware to provide the tables. Too few RMRRs are worse than too many. -- ljk > > Linus > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2015-04-27 16:09 ` Linda Knippers @ 2015-04-27 16:16 ` Alex Williamson 2015-04-27 17:28 ` David Woodhouse 2 siblings, 0 replies; 12+ messages in thread From: Alex Williamson @ 2015-04-27 16:16 UTC (permalink / raw) To: Linus Torvalds Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, David Woodhouse On Mon, 2015-04-27 at 08:48 -0700, Linus Torvalds wrote: > On Mon, Apr 27, 2015 at 8:39 AM, Alex Williamson > <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote: > > > > Since this got pulled anyway, do you plan to follow-up with patches to > > limit the graphics RMRR exception to the known and acceptable uses and > > document them, or should I send a revert patch? I don't think what we > > have here is acceptable going forward or being backported to stable. > > Thanks, > > Does anybody actually *care* about RMRR? > > The thing is shit. We know it's shit. We already had Linda pipe up to > tell us that there are bogus RMRR's for ever slot in older machines. > And we already ignore RMRR for USB controllers because it was just > useless garbage. > > Is there any reason to *not* just ignore RMRR for all video devices > like we do now? Seriously? > > I'm *so* not impressed with firmware tables. These things are always > uniformly wrong. We should strive to generate the information from our > actual hardware knowledge, with BIOS tables being the absolutely least > trusted source of information. It's true that there are RMRRs that are generated simply as placeholders on some systems and not actually used. However, it's also the case that some RMRRs are actively used and used in such a way that if we ignore them, we end up with corrupted VM memory because the data reporting done through the RMRR continues whether the IOVA is mapped to the intended reserved host address or memory backing the VM. From the kernel perspective, I don't see how we can tell the difference between placeholders and active RMRRs, nor do we really have any opportunity to introspect that this is occurring. In this data reporting mode, RMRRs are just a mechanism the platform can use to configure side-band data reporting between devices, outside of the scope or control of then kernel. I certainly don't know of any display devices that do this kind of data reporting, but I'm not thrilled about assuming they don't, or won't exist. Thanks, Alex ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2015-04-27 16:09 ` Linda Knippers 2015-04-27 16:16 ` Alex Williamson @ 2015-04-27 17:28 ` David Woodhouse 2 siblings, 0 replies; 12+ messages in thread From: David Woodhouse @ 2015-04-27 17:28 UTC (permalink / raw) To: Linus Torvalds; +Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA [-- Attachment #1.1: Type: text/plain, Size: 3948 bytes --] On Mon, 2015-04-27 at 08:48 -0700, Linus Torvalds wrote: > On Mon, Apr 27, 2015 at 8:39 AM, Alex Williamson > <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote: > > > > Since this got pulled anyway, do you plan to follow-up with > > patches to > > limit the graphics RMRR exception to the known and acceptable uses > > and > > document them, or should I send a revert patch? I don't think > > what we > > have here is acceptable going forward or being backported to > > stable. > > Thanks, > > Does anybody actually *care* about RMRR? > > The thing is shit. We know it's shit. We already had Linda pipe up to > tell us that there are bogus RMRR's for ever slot in older machines. > And we already ignore RMRR for USB controllers because it was just > useless garbage. RMRR for USB controllers was useless garbage because any operating system new enough to enable the IOMMU is *also* going to have USB support, so shouldn't need to rely on the BIOS doing legacy keyboard/mouse emulation. Which is the only reason the firmware would need to keep doing DMA from the USB controllers after we boot. They could have just said "quiesce the USB BIOS stuff before turning on the IOMMU" and they wouldn't have needed RMRR for USB. But even so, we don't just "ignore" the RMRR. We do set it the 1:1 mapping as requested, otherwise we'd get a stream of faults and the BIOS might even lock up in SMM code if its error handling is crap. The thing we *don't* do is faithfully preserve that 1:1 mapping for ever. If we assign the device to a VM guest, the RMRR mapping is lost. We've never preserved those mappings, and it's only ever been a problem on the insane crap that HP does with firmware-controlled DMA from stuff like storage controllers, which persists even at runtime after we're *using* said controller. (You might think it would have been easy enough for them to do their special DMA from another B-D-F but perhaps that would have been too easy for us.) It's extremely hard to preserve those mappings when assigning to a guest, because we'd actually have to poke a corresponding hole in the guest's physical address space (because in this case GPA==IOVA and those IOVAs are being used by the BIOS with a 1:1 mapping to HPA). Hence Alex's patch which blacklisted the 'unknown' RMRR-afflicted devices and prevented them from being assigned to guests (and also it seems from honouring 'iommu=pt'; qv.). > Is there any reason to *not* just ignore RMRR for all video devices > like we do now? Seriously? The video case is actually the *sanest* use of RMRR. With a framebuffer in main memory, the RMRR is the only thing that lets the graphics device continue to render pixels from the framebuffer after the IOMMU is initialised. So we *definitely* can't just 'ignore' the RMRR there. And we don't. But assignment of such graphics devices to guests *has* allegedly been working (and iommu=pt is kind of nice too because of the performance issues), and there's no reason for them to be excluded. They can be lumped in with the USB devices as "OK to assign to guests". We lose the 1:1 mapping indicated by the RMRR but that's OK because the guest has to initialise the hardware entirely for itself and set up its *own* framebuffer. > I'm *so* not impressed with firmware tables. These things are always > uniformly wrong. We should strive to generate the information from > our actual hardware knowledge, with BIOS tables being the absolutely > least trusted source of information. ... which is why the whole of the VT-d design, exposing this crap through firmware tables instead of hardware discoverability, makes me cringe. But while I am tilting at windmills internally to at least get people to realise why that's such a design flaw, we still have to deal with this crap. And with the insane things that HP abuse it for. -- dwmw2 [-- Attachment #1.2: smime.p7s --] [-- Type: application/x-pkcs7-signature, Size: 5745 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2015-04-27 15:48 ` Linus Torvalds @ 2015-04-27 17:12 ` David Woodhouse [not found] ` <1430154752.6377.19.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> 1 sibling, 1 reply; 12+ messages in thread From: David Woodhouse @ 2015-04-27 17:12 UTC (permalink / raw) To: Alex Williamson Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b [-- Attachment #1.1: Type: text/plain, Size: 1303 bytes --] On Mon, 2015-04-27 at 09:39 -0600, Alex Williamson wrote: > David, > > Since this got pulled anyway, do you plan to follow-up with patches to > limit the graphics RMRR exception to the known and acceptable uses and > document them, or should I send a revert patch? I don't think what we > have here is acceptable going forward or being backported to stable. > Thanks, I didn't plan to, no. So far we *only* know of acceptable uses of RMRR on graphics devices, and the false positives that Linda mentioned. The only cases we ever saw of RMRR being *problematic* when we tried to assign devices to guests were non-graphics devices, weren't they? Although obviously I'm looking into the oops that sl4ever reported; currently I'm wondering if that was broken even before your patch that blacklisted all RMRR-afflicted devices. It looks like device_to_iommu() doesn't return the correct results for devices behind a disabled IOMMU; it incorrectly attributes them to a catch-all IOMMU instead. Or perhaps this is probably tied into the fact that your patch broke iommu=pt for RMRR-afflicted devices too. They currently get full translation. Which surely shouldn't be necessary, since passthrough mode would obviously *also* preserve the RMRR regions of 1:1 mappings. -- dwmw2 [-- Attachment #1.2: smime.p7s --] [-- Type: application/x-pkcs7-signature, Size: 5745 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <1430154752.6377.19.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>]
* Re: [GIT PULL] Intel IOMMU for 4.1 [not found] ` <1430154752.6377.19.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> @ 2015-04-27 17:30 ` Alex Williamson 0 siblings, 0 replies; 12+ messages in thread From: Alex Williamson @ 2015-04-27 17:30 UTC (permalink / raw) To: David Woodhouse Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b On Mon, 2015-04-27 at 18:12 +0100, David Woodhouse wrote: > On Mon, 2015-04-27 at 09:39 -0600, Alex Williamson wrote: > > David, > > > > Since this got pulled anyway, do you plan to follow-up with patches to > > limit the graphics RMRR exception to the known and acceptable uses and > > document them, or should I send a revert patch? I don't think what we > > have here is acceptable going forward or being backported to stable. > > Thanks, > > I didn't plan to, no. So far we *only* know of acceptable uses of RMRR > on graphics devices, and the false positives that Linda mentioned. > > The only cases we ever saw of RMRR being *problematic* when we tried > to assign devices to guests were non-graphics devices, weren't they? That's true, but if we're only trying to enable IGD assignment (which of course doesn't work yet anyway), why add the entire PCI class rather than perhaps at least testing for vendor ID 8086 and pci_is_root_bus()? I don't see why we want to open the door any wider than necessary. > Although obviously I'm looking into the oops that sl4ever reported; > currently I'm wondering if that was broken even before your patch that > blacklisted all RMRR-afflicted devices. It looks like > device_to_iommu() doesn't return the correct results for devices > behind a disabled IOMMU; it incorrectly attributes them to a catch-all > IOMMU instead. > > Or perhaps this is probably tied into the fact that your patch broke > iommu=pt for RMRR-afflicted devices too. They currently get full > translation. Which surely shouldn't be necessary, since passthrough > mode would obviously *also* preserve the RMRR regions of 1:1 mappings. I think you're actually referring to: commit ea2447f700cab264019b52e2b417d689e052dcfd Author: Tom Mingarelli <thomas.mingarelli-VXdhtT5mjnY@public.gmane.org> Date: Tue Nov 20 19:43:17 2012 +0000 intel-iommu: Prevent devices with RMRRs from being placed into SI Domain This patch is to prevent non-USB devices that have RMRRs associated with the being placed into the SI Domain during init. This fixes the issue where the for devices being placed in and out of the SI Domain gets lost. Signed-off-by: Thomas Mingarelli <thomas.mingarelli-VXdhtT5mjnY@public.gmane.org> Tested-by: Shuah Khan <shuah.khan-VXdhtT5mjnY@public.gmane.org> Reviewed-by: Donald Dutile <ddutile-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> Reviewed-by: Alex Williamson <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> Signed-off-by: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> That patch has been present since v3.8 and was only cosmetically affected by my patch with a shared helper function. Thanks, Alex ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2015-04-27 17:30 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-04-26 8:51 [GIT PULL] Intel IOMMU for 4.1 David Woodhouse
[not found] ` <1430038266.4833.14.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-04-26 17:33 ` Alex Williamson
[not found] ` <1430069591.8301.91.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-26 17:55 ` David Woodhouse
[not found] ` <1430070924.13479.5.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-04-26 18:16 ` Alex Williamson
[not found] ` <1430072207.8301.103.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-27 14:00 ` Linda Knippers
[not found] ` <553E4109.5060903-VXdhtT5mjnY@public.gmane.org>
2015-04-27 15:39 ` Alex Williamson
[not found] ` <1430149195.4472.43.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2015-04-27 15:48 ` Linus Torvalds
[not found] ` <CA+55aFx+QhMZpCtnG2sHib+kk5BR+PeEa99-CCushSA5oAqVxw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-27 16:09 ` Linda Knippers
2015-04-27 16:16 ` Alex Williamson
2015-04-27 17:28 ` David Woodhouse
2015-04-27 17:12 ` David Woodhouse
[not found] ` <1430154752.6377.19.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-04-27 17:30 ` Alex Williamson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox