Re: Graphics pass-through

[Prasad Joshi <prasadjoshi124@gmail.com>]

On Mon, May 9, 2011 at 4:27 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2011-05-09 16:55, Prasad Joshi wrote:
>> On Mon, May 9, 2011 at 12:14 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>> On 2011-05-05 17:17, Alex Williamson wrote:
>>>>> And what about the host? When does Linux release the legacy range?
>>>>> Always or only when a specific (!=vga/vesa) framebuffer driver is loaded?
>>>>
>>>> Well, that's where it'd be nice if the vga arbiter was actually in more
>>>> widespread use.  It currently seems to be nothing more than a shared
>>>> mutex, but it would actually be useful if it included backends to do the
>>>> chipset vga routing changes.  I think when I was testing this, I was
>>>> externally poking PCI bridge chipset to toggle the VGA_EN bit.
>>>
>>> Right, we had to drop the approach to pass through the secondary card
>>> for now, the arbiter was not switching properly. Haven't checked yet if
>>> VGA_EN was properly set, though the kernel code looks like it should
>>> take care of this.
>>>
>>> Even with handing out the primary adapter, we had only mixed success so
>>> far. The onboard adapter worked well (in VESA mode), but the NVIDIA is
>>> not displaying early boot messages at all. Maybe a vgabios issue.
>>> Windows was booting nevertheless - until we installed the NVIDIA
>>> drivers. Than it ran into a blue screen.
>>>
>>> BTW, what ATI adapter did you use precisely, and what did work, what not?
>>
>> Not hijacking the mail thread. Just wanted to provide some inputs.
>
> Much appreciated in fact!
>
>>
>> Few days back I had tried passing through the secondary graphics card.
>> I could pass-through two graphics cards to virtual machine.
>>
>> 02:00.0 VGA compatible controller: ATI Technologies Inc Redwood
>> [Radeon HD 5670] (prog-if 00 [VGA controller])
>>       Subsystem: PC Partner Limited Device e151
>>       Flags: bus master, fast devsel, latency 0, IRQ 87
>>       Memory at d0000000 (64-bit, prefetchable) [size=256M]
>>       Memory at fe6e0000 (64-bit, non-prefetchable) [size=128K]
>>       I/O ports at b000 [size=256]
>>       Expansion ROM at fe6c0000 [disabled] [size=128K]
>>       Capabilities: <access denied>
>>       Kernel driver in use: radeon
>>       Kernel modules: radeon
>>
>> 07:00.0 VGA compatible controller: nVidia Corporation G86 [Quadro NVS
>> 290] (rev a1) (prog-if 00 [VGA controller])
>>        Subsystem: nVidia Corporation Device 0492
>>        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>> ParErr-Stepping- SERR+ FastB2B- DisINTx-
>>        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
>>> TAbort-<TAbort- <MAbort- >SERR- <PERR- INTx-
>>        Latency: 0, Cache Line Size: 64 bytes
>>        Interrupt: pin A routed to IRQ 24
>>        Region 0: Memory at fd000000 (32-bit, non-prefetchable) [size=16M]
>>        Region 1: Memory at d0000000 (64-bit, prefetchable) [size=256M]
>>        Region 3: Memory at fa000000 (64-bit, non-prefetchable) [size=32M]
>>        Region 5: I/O ports at ec00 [size=128]
>>        Expansion ROM at fe9e0000 [disabled] [size=128K]
>>        Capabilities: <access denied>
>>        Kernel driver in use: nouveau
>>        Kernel modules: nouveau, nvidiafb
>>
>> Both of them are PCIe cards. I have one more ATI card and another
>> NVIDIA card which does not work.
>
> Interesting. That may rule out missing PCIe capabilities as source for
> the NVIDIA driver indisposition.
>
> Did you passed those cards each as primary to the guest, or was the
> guest seeing multiple adapters?

I passed the graphics device as a primary device to the guest virtual
machine, with -vga none parameter to disable the default vga device.

> I presume you only got output after
> early boot was completed, right?

Yes you are correct. I got the display, only after the KMS was
started. The initial BIOS messages were not displayed.

>
> To avoid having to deal with legacy I/O forwarding, we started with a
> dual adapter setup in the hope to leave the primary guest adapter at
> know-to-work cirrus-vga. But already in a native setup with on-board
> primary + NVIDIA secondary, the NVIDIA Windows drivers refused to talk
> to its hardware in this constellation.
>

Windows operating system never worked for me with either of the graphics card.

>>
>> One of the reason the pass-through did not work is because of the
>> limit on amount of pci configuration memory by SeaBIOS. SeaBIOS places
>> a hard limit of 256MB or so on the amount of PCI memory space. Thus,
>> for some of the VGA device that need more memory never worked for me.
>>
>> SeaBIOS allows this memory region to be extended to some value near
>> 512MB, but even then the range is not enough.
>>
>> Another problem with SeaBIOS which limits the amount of memory space
>> is: SeaBIOS allocates the BAR regions as they are encountered. As far
>> as I know, the BAR regions should be naturally aligned. Thus the
>> simple strategy of the SeaBIOS results in large fragmentation.
>> Therefore, even after increasing the PCI memory space to 512MB the BAR
>> regions were unallocated.
>
> That's an interesting trace! We'll check this here, but I bet it
> contributes to the problems. Our FX 3800 has 1G memory...

Yes it is one of the problem. I remember reading something about the
NVIDIA BIOS and FLR, those could be other interesting issues.

>
>>
>> I will confirm you the details of other graphics cards which do not work.
>
> TiA,
> Jan
>
> --
> Siemens AG, Corporate Technology, CT T DE IT 1
> Corporate Competence Center Embedded Linux
>