Re: IOMMU Domain for Dom0

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Sun, Jun 26, 2011 at 09:35:17PM -0600, Alex Merritt wrote:
> Hello,
> 
> I'm looking to enable the NVIDIA CUDA driver/runtime stack to work in
> Dom0 on Xen. I've contacted NVIDIA through various capacities and have

I've been running the NVIDIA binary driver with 2.6.39 / 3.0 kernel.

> gotten replies essentially saying they cannot provide assistance, and
> have been following the nvnews.com forums. However, now that I have
> IOMMU-capable processors (with VT-d) and a version of Xen which can
> successfully program this hardware, I am interested to determine if it
> is possible to program the IOMMU *for* Dom0 (i.e. the target is Dom0,
> not an HVM/pvops guest).
> 
> I have successfully been able to launch an HVM guest, passing a few
> GPUs through. Installing an unmodified developer driver from NVIDIA
> works using the standard method (no SYSSRC=/-OUT= or anything) and
> CUDA applications execute as expected. Can this be done for Dom0,
> achieving the same result without requiring any involvement from
> NVIDIA or modifications to the driver?

It should be no trouble.. albeit you might need to update the Linux
kernel to 2.6.39 or 3.0 to take advantage of the 1-1 P2M mapping code.

> 
> My immediate interest is more to see if it "can be done" via a hack or
> something, not necessarily whether it would make sense for Xen to
> support this in the future. My goal with this email is to get feedback
> on two fronts: 1) is there a limitation due to the architecture,
> meaning that as Dom0 is pvops it cannot use VT-d, or is it a
> implementation addition to Xen that would be needed? 2) If the latter
> is true (programming needed), how much effort (code/time) would
> you/anyone estimate it would take to enable this to work, and could
> you provide some starting pointers for me to do so? I am untrained
> when it comes to the Xen sourcecode.
> 
> I'm using Xen 4.1.1 and pv-ops linux (not upstream) 2.6.32.40 on an
> Intel X5660 with a Tylersburg chipset. My host OS is Fedora 13, but
> that needn't be static.
> 
> I originally asked this on xen-users, but was informed this mailing
> list would be better suited:
> http://lists.xensource.com/archives/html/xen-users/2011-06/msg00451.html
> 
> Many thanks in advance!
> Alex
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel