[PATCH] iommu/intel: Exclude devices using RMRRs from IOMMU API domains

[PATCH] iommu/intel: Exclude devices using RMRRs from IOMMU API domains

[Alex Williamson]

The user of the IOMMU API domain expects to have full control of
the IOVA space for the domain.  RMRRs are fundamentally incompatible
with that idea.  We can neither map the RMRR into the IOMMU API
domain, nor can we guarantee that the device won't continue DMA with
the area described by the RMRR as part of the new domain.  Therefore
we must prevent such devices from being used by the IOMMU API.

Signed-off-by: Alex Williamson <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 drivers/iommu/intel-iommu.c |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f256ffc..eb77149 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -4188,6 +4188,21 @@ static int intel_iommu_attach_device(struct iommu_domain *domain,
 	int addr_width;
 	u8 bus, devfn;
 
+	/*
+	 * With IOMMU API domains we don't have the freedom to insert RMRR
+	 * entries into the domain mapping, the IOMMU API user expects full
+	 * control of the IOVA space of the device.  We also have no ability
+	 * to shutdown whatever back channel operations occur through the
+	 * RMRR.  Therefore our only option is to prevent devices making use
+	 * of RMRRs from being used by the IOMMU API.  As usual we exempt
+	 * USB devices since their RMRR support is largely historical.
+	 */
+	if (device_has_rmrr(dev) && (!dev_is_pci(dev) ||
+	    (to_pci_dev(dev)->class >> 8) != PCI_CLASS_SERIAL_USB)) {
+		dev_warn(dev, "Device is ineligible for IOMMU domain attach due to platform RMRR requirement.  Contact your platform vendor.\n");
+		return -EPERM;
+	}
+
 	/* normally dev is not mapped */
 	if (unlikely(domain_context_mapped(dev))) {
 		struct dmar_domain *old_domain;

Re: [PATCH] iommu/intel: Exclude devices using RMRRs from IOMMU API domains

[David Woodhouse]

[-- Attachment #1.1: Type: text/plain, Size: 2414 bytes --]

On Wed, 2014-05-14 at 13:27 -0600, Alex Williamson wrote:
> The user of the IOMMU API domain expects to have full control of
> the IOVA space for the domain.  RMRRs are fundamentally incompatible
> with that idea.  We can neither map the RMRR into the IOMMU API
> domain, nor can we guarantee that the device won't continue DMA with
> the area described by the RMRR as part of the new domain.  Therefore
> we must prevent such devices from being used by the IOMMU API.

Ick, ick, ick. The more the ramifications of RMRRs become apparent, the
more I wish we'd just done the Right Thing™ and declared that firmware
SHALL NOT leave any device doing (IOMMU-visible) DMA after the OS takes
over. That way, if they wanted this kind of abomination then they'd have
to come up with a way of doing it differently. Hell, can't you do PCIe
transactions which claim to be already translated, and thus just bypass
the IOMMU?

OK, rant over... 

Why can't we map the RMRR into the IOMMU API domain? If we're setting up
a VM guest, that basically means we'd want to poke a hole in its memory
map and mark the RMRR-afflicted range as reserved or absent. It's
horrible, but *everything* about RMRRs is horrible. It's not impossible,
and it would allow us to give these devices away to guests. Don't we
sometimes *have* devices that we want to give to guests, that are
afflicted with RMRRs?

There are discussions about RMRRs being (ab)used for more than their
existing brain-damaged purpose. Where we have a peripheral device that
will (mis)interpret certain address ranges as "local" rather than
forwarding transactions up towards main memory, we need to ensure that
such ranges are never used as virtual addresses. This has largely been
an invisible problem until we found a device where the affected range
matched the IOVA our DMA API uses by default. Using an RMRR has been
proposed as a simple way to achieve that... which means that you end up
not being able to assign *those* devices to IOMMU domains either.

I do suspect it's going to lead to complaints... but I'm just not sure I
can bring myself to care. Sane designs don't require RMRRs. If someone
comes to me and complains that their HP storage controller or whatever
can't be assigned to a guest, I'm quite prepared to tell them to replace
it with something non-broken. Will you back me up when it happens?

-- 
dwmw2


[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH] iommu/intel: Exclude devices using RMRRs from IOMMU API domains

[Alex Williamson]

On Wed, 2014-05-21 at 11:38 +0100, David Woodhouse wrote:
> On Wed, 2014-05-14 at 13:27 -0600, Alex Williamson wrote:
> > The user of the IOMMU API domain expects to have full control of
> > the IOVA space for the domain.  RMRRs are fundamentally incompatible
> > with that idea.  We can neither map the RMRR into the IOMMU API
> > domain, nor can we guarantee that the device won't continue DMA with
> > the area described by the RMRR as part of the new domain.  Therefore
> > we must prevent such devices from being used by the IOMMU API.
> 
> Ick, ick, ick. The more the ramifications of RMRRs become apparent, the
> more I wish we'd just done the Right Thing™ and declared that firmware
> SHALL NOT leave any device doing (IOMMU-visible) DMA after the OS takes
> over. That way, if they wanted this kind of abomination then they'd have
> to come up with a way of doing it differently. Hell, can't you do PCIe
> transactions which claim to be already translated, and thus just bypass
> the IOMMU?
> 
> OK, rant over... 
> 
> Why can't we map the RMRR into the IOMMU API domain? If we're setting up
> a VM guest, that basically means we'd want to poke a hole in its memory
> map and mark the RMRR-afflicted range as reserved or absent. It's
> horrible, but *everything* about RMRRs is horrible. It's not impossible,
> and it would allow us to give these devices away to guests. Don't we
> sometimes *have* devices that we want to give to guests, that are
> afflicted with RMRRs?

You're right, it is possible to assign devices with RMRRs, but in order
to do so we'd need to expose the RMRR areas for a device beyond the
inner workings of intel-iommu and mark those ranges as reserved in the
guest.  That alone makes hotplug of such devices into a guest
impossible.

Enabling such a use case also potentially provides an untrusted guest
with direct access to regions of platform memory that potentially allows
for untold platform specific exploits.

We currently have no visibility to RMRRs from the IOMMU API, so we can't
even attempt to do the above, nor can we guarantee that we have any
ability to make a device discontinue use of an RMRR area when it is
assigned to a VM domain.  Therefore the only safe thing to do is prevent
use of such devices by a VM domain.

> There are discussions about RMRRs being (ab)used for more than their
> existing brain-damaged purpose. Where we have a peripheral device that
> will (mis)interpret certain address ranges as "local" rather than
> forwarding transactions up towards main memory, we need to ensure that
> such ranges are never used as virtual addresses. This has largely been
> an invisible problem until we found a device where the affected range
> matched the IOVA our DMA API uses by default. Using an RMRR has been
> proposed as a simple way to achieve that... which means that you end up
> not being able to assign *those* devices to IOMMU domains either.
> 
> I do suspect it's going to lead to complaints... but I'm just not sure I
> can bring myself to care. Sane designs don't require RMRRs. If someone
> comes to me and complains that their HP storage controller or whatever
> can't be assigned to a guest, I'm quite prepared to tell them to replace
> it with something non-broken. Will you back me up when it happens?

Exactly, I have a hard time bringing myself to care about supporting
such devices.  Vendors that proliferate RMRR usage need to be aware of
the implications of RMRRs for all use cases of a device.  First and
foremost, we need to lock out devices with RMRRs because we have no
ability to either honor or disable RMRRs when used by the IOMMU API.  If
vendors that rely on RMRRs want to make such devices assignable by
providing interfaces to describe and map the area into a VM, or even a
mechanism to disable the RMRR, more power to them.  The current
situation is simply unsafe and needs to be prevented.  Thanks,

Alex

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH] iommu/intel: Exclude devices using RMRRs from IOMMU API domains

[Linda Knippers]

On 5/21/2014 9:20 AM, Alex Williamson wrote:
> On Wed, 2014-05-21 at 11:38 +0100, David Woodhouse wrote:
>> On Wed, 2014-05-14 at 13:27 -0600, Alex Williamson wrote:
>>> The user of the IOMMU API domain expects to have full control of
>>> the IOVA space for the domain.  RMRRs are fundamentally incompatible
>>> with that idea.  We can neither map the RMRR into the IOMMU API
>>> domain, nor can we guarantee that the device won't continue DMA with
>>> the area described by the RMRR as part of the new domain.  Therefore
>>> we must prevent such devices from being used by the IOMMU API.
>>
>> Ick, ick, ick. The more the ramifications of RMRRs become apparent, the
>> more I wish we'd just done the Right Thing™ and declared that firmware
>> SHALL NOT leave any device doing (IOMMU-visible) DMA after the OS takes
>> over. That way, if they wanted this kind of abomination then they'd have
>> to come up with a way of doing it differently. Hell, can't you do PCIe
>> transactions which claim to be already translated, and thus just bypass
>> the IOMMU?
>>
>> OK, rant over... 
>>
>> Why can't we map the RMRR into the IOMMU API domain? If we're setting up
>> a VM guest, that basically means we'd want to poke a hole in its memory
>> map and mark the RMRR-afflicted range as reserved or absent. It's
>> horrible, but *everything* about RMRRs is horrible. It's not impossible,
>> and it would allow us to give these devices away to guests. Don't we
>> sometimes *have* devices that we want to give to guests, that are
>> afflicted with RMRRs?
> 
> You're right, it is possible to assign devices with RMRRs, but in order
> to do so we'd need to expose the RMRR areas for a device beyond the
> inner workings of intel-iommu and mark those ranges as reserved in the
> guest.  That alone makes hotplug of such devices into a guest
> impossible.
> 
> Enabling such a use case also potentially provides an untrusted guest
> with direct access to regions of platform memory that potentially allows
> for untold platform specific exploits.
> 
> We currently have no visibility to RMRRs from the IOMMU API, so we can't
> even attempt to do the above, nor can we guarantee that we have any
> ability to make a device discontinue use of an RMRR area when it is
> assigned to a VM domain.  Therefore the only safe thing to do is prevent
> use of such devices by a VM domain.
> 
>> There are discussions about RMRRs being (ab)used for more than their
>> existing brain-damaged purpose. Where we have a peripheral device that
>> will (mis)interpret certain address ranges as "local" rather than
>> forwarding transactions up towards main memory, we need to ensure that
>> such ranges are never used as virtual addresses. This has largely been
>> an invisible problem until we found a device where the affected range
>> matched the IOVA our DMA API uses by default. Using an RMRR has been
>> proposed as a simple way to achieve that... which means that you end up
>> not being able to assign *those* devices to IOMMU domains either.
>>
>> I do suspect it's going to lead to complaints... but I'm just not sure I
>> can bring myself to care. Sane designs don't require RMRRs. If someone
>> comes to me and complains that their HP storage controller or whatever
>> can't be assigned to a guest, I'm quite prepared to tell them to replace
>> it with something non-broken. Will you back me up when it happens?
> 
> Exactly, I have a hard time bringing myself to care about supporting
> such devices.  Vendors that proliferate RMRR usage need to be aware of
> the implications of RMRRs for all use cases of a device.  First and
> foremost, we need to lock out devices with RMRRs because we have no
> ability to either honor or disable RMRRs when used by the IOMMU API.  If
> vendors that rely on RMRRs want to make such devices assignable by
> providing interfaces to describe and map the area into a VM, or even a
> mechanism to disable the RMRR, more power to them.  The current
> situation is simply unsafe and needs to be prevented.  

I care but I think this patch should go in until there is a better
solution.

-- ljk

> Thanks,
> 
> Alex
> 
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu
> 

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH] iommu/intel: Exclude devices using RMRRs from IOMMU API domains

[David Woodhouse]

[-- Attachment #1.1: Type: text/plain, Size: 1208 bytes --]

On Wed, 2014-05-21 at 13:05 -0400, Linda Knippers wrote:
> 
> > Exactly, I have a hard time bringing myself to care about supporting
> > such devices.  Vendors that proliferate RMRR usage need to be aware of
> > the implications of RMRRs for all use cases of a device.  First and
> > foremost, we need to lock out devices with RMRRs because we have no
> > ability to either honor or disable RMRRs when used by the IOMMU API.  If
> > vendors that rely on RMRRs want to make such devices assignable by
> > providing interfaces to describe and map the area into a VM, or even a
> > mechanism to disable the RMRR, more power to them.  The current
> > situation is simply unsafe and needs to be prevented.  
> 
> I care but I think this patch should go in until there is a better
> solution.

I was quite happy with our earlier "solution" of just assuming that
RMRRs were a boot-time thing only. Once a driver takes over the device,
or it's given to a guest, we forget the RMRRs entirely. The hardware has
no business still doing rogue DMA after we've taken control of it.

Again, it's only those who stupidly designed hardware who suffer. And
life was a lot simpler for us.

-- 
dwmw2


[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]