Re: Some trouble to use NVIDIA CUDA with Xen

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Mon, Aug 12, 2013 at 01:33:13PM +0100, Gordan Bobic wrote:
> Your Nvidia device ID seems to imply a GTX770.
> I could be wrong, but wasn't this only supported on Quadro/Grid GPUs?
> Or is that limitation only applicable to Windows?
> 
> Gordan
> 
> On Sat, 10 Aug 2013 18:21:32 +0200 (CEST), Martin Cerveny
> <martin@c-home.cz> wrote:
> >Hello.
> >
> >Any progress on this topics ?
> >What is the blocker to run correctly NVIDIA (proprietary) drivers in
> >kernel in XEN/Dom0 ?
> >
> >The problem still persit:
> >
> >- CUDA "deviceQuery" run and exit without error (BUT with xen after
> >9sec, without xen after 1sec)
> >- CUDA other programs for example "bandwidthTest" exit with error
> >"code=46(cudaErrorDevicesUnavailable)"
> >
> >Test environment:
> >
> >- fedora18
> >- kernel 3.9.11-200.fc18.x86_64
> >- nvidia drivers 319.37 (comes with CUDA)
> >- nvidia CUDA 5.5
> >- XEN 4.2.2 and directly from git repo 4.4.unstable (commit
> >73f18583dd824f0e49f65149ef603600ce31b8ee)
> >- AMD Athlon(tm) 64 X2 Dual Core Processor 5600+
> >
> >Attached files:
> >
> >- output from dmesg, deviceQuery (with /bin.time), bandwidthTest
> >(with /bin/time), lspci -vvv
> >
> >Hypotehesis:
> >
> >- PCI output are the same - probably no problem
> >- dmesg differences ( diff dmesg_boot_without_xen_wt.txt
> >dmesg_boot_xen_44u_wt.txt )
> >
> >======================
> >without XEN:
> >
> >< MTRR default type: uncachable
> >< MTRR fixed ranges enabled:
> ><   00000-9FFFF write-back
> ><   A0000-EFFFF uncachable
> ><   F0000-FFFFF write-protect
> >< MTRR variable ranges enabled:
> ><   0 base 0000000000 mask FF80000000 write-back
> ><   1 base 0080000000 mask FFC0000000 write-back
> ><   2 disabled
> ><   3 disabled
> ><   4 disabled
> ><   5 disabled
> ><   6 disabled
> ><   7 disabled
> >< x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
> >
> >======================
> >with XEN:
> >
> >>NVRM: PAT configuration unsupported.

Right, so there are couple of patches that can enable that back.


You need to revert these two:
8eaffa67b43e99ae581622c5133e20b0f48bcef1
c79c49826270b8b0061b2fca840fc3f013c8a78a

And apply this patch:

https://lkml.org/lkml/2012/2/10/229

That should re-enable PAT.  Try that and please report back.
> >
> >======================
> >
> >The MTRR+PAT is not supported (still) in kernel in XEN/dom0 ?
> >Maybe MTRR+PAT is needed for CUDA too.
> >
> >Is there any workaround for linux kernels ~ 3.9.x and xen ~ v4.x.x ?
> >
> >Thanks, Martin Cerveny
> >
> >Refs:
> >
> >[Xen-devel] XEN MTRR -
> >http://lists.xen.org/archives/html/xen-devel/2012-06/msg00194.html
> >[Xen-users] Xen and nvidia -
> >http://lists.xen.org/archives/html/xen-users/2013-01/msg00169.html
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel