Re: Nvidia, Xen, and Vt-d

[Michael J Coss <mjcoss@alcatel-lucent.com>]

Jeremy Fitzhardinge wrote:
> On 09/09/09 01:56, Keir Fraser wrote:
>   
>> On 09/09/2009 09:47, "Michael J Coss" <mjcoss@alcatel-lucent.com> wrote:
>>
>>   
>>     
>>> I've tried 3.4.1, and the lastest xen-unstable.  I've tried the pv-ops
>>> git tree as I really need a 2.6.31 dom0 kernel for other reasons, but
>>> for the moment I'd just like to get to the point where I have Xen, and
>>> dom0 and Nvidia playing nicely with one another, so I can move on to
>>> working on Vt-d and graphic pass through.
>>>
>>> Any suggestions?
>>>     
>>>       
>> Unless you really must have 2.6.30+, I'd recommend the 2.6.27 tree and
>> patchqueue from http://xenbits.xensource.com/XCI. Otherwise you are likely
>> to have to get your hands fairly dirty with pv_ops. For example, afaik
>> starting an X server on pv_ops is still pretty ambitious on some systems.
>>   
>>     
>
> Starting X in dom0 seems to work OK for Intel and ATI systems, at least;
> I expect most DRM drivers would work OK if they're well-behaved because
> we're hooking AGP memory accesses, etc.  However, the proprietary Nvidia
> drivers are problematic, though I gather there are some patches floating
> around for them.
>
> Unfortunately the AGP hooks are being removed (some years after Keir
> first added them, and just as they have a user according to their
> original intent) in favour of making each driver use the DMA API to do
> the appropriate phys<->bus conversions.  So far, only the Intel driver
> has been converted, and only when Intel IOMMU is enabled.  However, I
> didn't get any objection from the DRM folks about making it
> unconditional or adding it to new drivers as needed.
>
>     J
>   
I suspected as much, although I don't understand the origin of the lspci 
discrepancies between booting with/without the hypervisor.  It seems to 
me that there is some problem with Xen's view of the PCI bus, as well as 
the fact that the Nvidia driver is trying to access something outside of 
the hooked APIs.  The graphics cards in the system are the dual GPU, 
dual slot cards, and maybe this is contributing to the problem.  I'm 
going to see about getting some other single slot Nvidia card and see if 
the same issue happens.  I may pick up some ATI cards as well.

---Michael J Coss