Re: Device specific pass through in host systems - discuss user interface

From: Robin Murphy <robin.murphy@arm.com>
To: "Prakhya, Sai Praneeth" <sai.praneeth.prakhya@intel.com>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>
Cc: "Shankar, Ravi V" <ravi.v.shankar@intel.com>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"jroedel@suse.de" <jroedel@suse.de>,
	"Raj, Ashok" <ashok.raj@intel.com>,
	Will Deacon <will.deacon@arm.com>,
	"Pan, Jacob jun" <jacob.jun.pan@intel.com>,
	"hch@lst.de" <hch@lst.de>, "Lu, Baolu" <baolu.lu@intel.com>
Subject: Re: Device specific pass through in host systems - discuss user interface
Date: Fri, 7 Jun 2019 13:41:14 +0100	[thread overview]
Message-ID: <3fa16135-b2bb-03b5-833a-bce8cc68f35f@arm.com> (raw)
In-Reply-To: <FFF73D592F13FD46B8700F0A279B802F48DA796E@ORSMSX114.amr.corp.intel.com>

On 07/06/2019 03:24, Prakhya, Sai Praneeth wrote:
> Hi All,
> 
> I am working on an IOMMU driver feature that allows a user to specify if the DMA from a device should be translated by IOMMU or not. Presently, we support only all devices or none mode i.e. if user specifies "iommu=pt" [X86] or "iommu.passthrough" [ARM64] through kernel command line, all the devices would be in pass through mode and we don't have per device granularity, but, we were requested by a customer to selectively put devices in pass through mode and not all.

It's interesting to see this from a fresh angle which isn't clouded by 
other SoC GPU details - thanks for the proposal! A couple more thoughts 
jump out immediately...

> Since, this feature could be generic across architectures, we thought it would be better if the user interface is discussed in the community first. We are envisioning this to be used both during boot time and runtime and hence having a kernel command line argument along with a sysfs entry are needed. So, please pour in your suggestions on how the user interface should look like to make it architecture agnostic.
> 
> 
> 1.      Have a kernel command line argument that takes a list of BDF's as an input and puts them in pass through mode
> 
> a.      Accepting BDF as an input has a downside - BDF is dynamic and could change if BIOS/OS enumerates a new device in next reboot
> 
> b.      Accepting <vendor_id:device_id> pair as an input has a downside - What to do when there are multiple such devices and user would like to put only some of them in PT mode
> 

c. Not all devices are PCI in the first place.

> 2.      Have a sysfs file which takes 1 or 0 as an input to enable/disable pass through mode. Some places that seem to be reasonable are
> 
> a.      /sys/class/iommu/dmar0/devices/
> 
> b.      /sys/kernel/iommu_groups/<id>/devices

Note that this this works out a bit tricky to actually use, since 
changing the meaning of DMA addresses under the device's feet would be 
disastrous. It can only safely take effect by unbinding and rebinding 
the driver and/or resetting the device (as a side note, this starts to 
overlap with the IOMMU-drivers-as-modules concept, and whatever solution 
to this we end up with may be helpful in making that work too). In most 
cases that's probably viable, but not every driver supports unbinding, 
and either way what if the device in question is the one hosting the 
root filesystem... :/

Robin.

> 
> I am looking for a consensus on *how the kernel command line argument should look like and path for sysfs entry*. Also, please note that if a device is put in pass through mode it won't be available for the guest and that's ok.
> 
> Regards,
> Sai
> 
> PS: Idea credits: Ashok Raj
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu