Re: Success with pv guest and ATI Radeon HD4550

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 08/12/15 14:55, Geoffrey McRae wrote:
>
> On 09/12/15 01:51, Andrew Cooper wrote:
>> On 08/12/15 14:37, Geoffrey McRae wrote:
>>> Hi All,
>>>
>>> This is just a heads up on the steps involved in getting a HD4550
>>> working with the ATI driver as a pv guest on a Dom0 with no IOMMU
>>> support. The main issue with making this work was providing a way for
>>> the guest to read the vga bios.
>>>
>>> [    2.127078] [drm] Initialized drm 1.1.0 20060810
>>> [    2.140666] snd_hda_intel 0000:00:00.1: Xen PCI mapped GSI19 to
>>> IRQ51
>>> [    2.140950] snd_hda_intel 0000:00:00.1: Handle VGA-switcheroo audio
>>> client
>>> [    2.143196] [drm] radeon kernel modesetting enabled.
>>> [    2.143856] radeon 0000:00:00.0: Xen PCI mapped GSI18 to IRQ53
>>> [    2.144583] [drm] initializing kernel modesetting (RV710
>>> 0x1002:0x9540 0x1043:0x0298).
>>> [    2.144608] [drm] register mmio base: 0xFBDF0000
>>> [    2.144612] [drm] register mmio size: 65536
>>> [    2.144693] radeon 0000:00:00.0: Invalid ROM contents
>>> [    2.144703] radeon 0000:00:00.0: Invalid ROM contents
>>> [    2.144707] [drm:radeon_get_bios] *ERROR* Unable to locate a BIOS
>>> ROM
>>> [    2.144712] radeon 0000:00:00.0: Fatal error during GPU init
>>> [    2.144717] [drm] radeon: finishing device.
>>> [    2.144720] [TTM] Memory type 2 has not been initialized
>>>
>>> To resolve this I dumped the BIOS on dom0 to a file and copied it into
>>> the guest, then patched radeon_bios.c to provide a means to load it
>>> from userspace with the following result.
>>>
>>>
>>> [  997.220370] [drm] radeon kernel modesetting enabled.
>>> [  997.220743] xen:events: xen_bind_pirq_gsi_to_irq: returning irq 53
>>> for gsi 18
>>> [  997.220753] radeon 0000:00:00.0: Xen PCI mapped GSI18 to IRQ53
>>> [  997.221304] [drm] initializing kernel modesetting (RV710
>>> 0x1002:0x9540 0x1043:0x0298).
>>> [  997.221329] [drm] register mmio base: 0xFBDF0000
>>> [  997.221333] [drm] register mmio size: 65536
>>> [  997.222605] radeon 0000:00:00.0: firmware: direct-loading firmware
>>> radeon.bios
>>> [  997.222652] ATOM BIOS: 9540.11.17.0.18.AS02
>>> [  997.222691] radeon 0000:00:00.0: VRAM: 512M 0x0000000000000000 -
>>> 0x000000001FFFFFFF (512M used)
>>> [  997.222702] radeon 0000:00:00.0: GTT: 1024M 0x0000000020000000 -
>>> 0x000000005FFFFFFF
>>> [  997.222708] Failed to add WC MTRR for
>>> [00000000d0000000-00000000dfffffff]; performance may suffer.
>>> [  997.222713] [drm] Detected VRAM RAM=512M, BAR=256M
>>> [  997.222717] [drm] RAM width 64bits DDR
>>> [  997.222815] [TTM] Zone  kernel: Available graphics memory: 509996
>>> kiB
>>> [  997.222821] [TTM] Initializing pool allocator
>>> [  997.222829] [TTM] Initializing DMA pool allocator
>>> [  997.222862] [drm] radeon: 512M of VRAM memory ready
>>> [  997.222866] [drm] radeon: 1024M of GTT memory ready.
>>> [  997.222884] [drm] Loading RV710 Microcode
>>> [  997.231622] radeon 0000:00:00.0: firmware: direct-loading firmware
>>> radeon/RV710_pfp.bin
>>> [  997.249253] radeon 0000:00:00.0: firmware: direct-loading firmware
>>> radeon/RV710_me.bin
>>> [  997.272966] radeon 0000:00:00.0: firmware: direct-loading firmware
>>> radeon/R700_rlc.bin
>>> [  997.290901] radeon 0000:00:00.0: firmware: direct-loading firmware
>>> radeon/RV710_smc.bin
>>> [  997.290915] [drm] Internal thermal controller without fan control
>>> [  997.291866] [drm] radeon: dpm initialized
>>> [  997.306236] radeon 0000:00:00.0: firmware: direct-loading firmware
>>> radeon/RV710_uvd.bin
>>> [  997.306409] [drm] GART: num cpu pages 262144, num gpu pages 262144
>>> [  997.320178] [drm] PCIE GART of 1024M enabled (table at
>>> 0x000000000025D000).
>>> [  997.320240] radeon 0000:00:00.0: WB enabled
>>> [  997.320246] radeon 0000:00:00.0: fence driver on ring 0 use gpu
>>> addr 0x0000000020000c00 and cpu addr 0xffff88003c79cc00
>>> [  997.320253] radeon 0000:00:00.0: fence driver on ring 3 use gpu
>>> addr 0x0000000020000c0c and cpu addr 0xffff88003c79cc0c
>>> [  997.323053] radeon 0000:00:00.0: fence driver on ring 5 use gpu
>>> addr 0x000000000005c598 and cpu addr 0xffffc9000031c598
>>> [  997.323060] [drm] Supports vblank timestamp caching Rev 2
>>> (21.10.2013).
>>> [  997.323063] [drm] Driver supports precise vblank timestamp query.
>>> [  997.323067] radeon 0000:00:00.0: radeon: MSI limited to 32-bit
>>> [  997.323285] radeon 0000:00:00.0: radeon: using MSI.
>>> [  997.323322] [drm] radeon: irq initialized.
>>> [  997.369642] [drm] ring test on 0 succeeded in 1 usecs
>>> [  997.369651] [drm] ring test on 3 succeeded in 2 usecs
>>> [  997.564145] [drm] ring test on 5 succeeded in 1 usecs
>>> [  997.564153] [drm] UVD initialized successfully.
>>> [  997.564580] [drm] ib test on ring 0 succeeded in 0 usecs
>>> [  997.564614] [drm] ib test on ring 3 succeeded in 0 usecs
>>> [  997.723870] [drm] ib test on ring 5 succeeded
>>> [  997.724326] [drm] Radeon Display Connectors
>>> [  997.724332] [drm] Connector 0:
>>> [  997.724335] [drm]   VGA-1
>>> [  997.724338] [drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48
>>> 0x7e4c 0x7e4c
>>> [  997.724342] [drm]   Encoders:
>>> [  997.724345] [drm]     CRT2: INTERNAL_KLDSCP_DAC2
>>> [  997.724348] [drm] Connector 1:
>>> [  997.724350] [drm]   HDMI-A-1
>>> [  997.724353] [drm]   HPD1
>>> [  997.724355] [drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58
>>> 0x7e5c 0x7e5c
>>> [  997.724359] [drm]   Encoders:
>>> [  997.724362] [drm]     DFP1: INTERNAL_UNIPHY
>>> [  997.724365] [drm] Connector 2:
>>> [  997.724367] [drm]   DVI-I-1
>>> [  997.724369] [drm]   HPD4
>>> [  997.724372] [drm]   DDC: 0x7f10 0x7f10 0x7f14 0x7f14 0x7f18 0x7f18
>>> 0x7f1c 0x7f1c
>>> [  997.724376] [drm]   Encoders:
>>> [  997.724378] [drm]     CRT1: INTERNAL_KLDSCP_DAC1
>>> [  997.724382] [drm]     DFP2: INTERNAL_UNIPHY2
>>> [  997.779239] [drm] fb mappable at 0xD045E000
>>> [  997.779245] [drm] vram apper at 0xD0000000
>>> [  997.779251] [drm] size 5242880
>>> [  997.779253] [drm] fb depth is 24
>>> [  997.779255] [drm]    pitch is 5120
>>> [  997.779366] fbcon: radeondrmfb (fb0) is primary device
>>> [  997.792290] Console: switching to colour frame buffer device 160x64
>>> [  997.805919] radeon 0000:00:00.0: fb0: radeondrmfb frame buffer
>>> device
>>> [  997.806021] radeon 0000:00:00.0: registered panic notifier
>>> [  997.820118] [drm] Initialized radeon 2.39.0 20080528 for
>>> 0000:00:00.0 on minor 0
>>>
>>> The changes made were in `drivers/gpu/drm/radeon/radeon_bios.c` where
>>> I added the following method, and a call to it in `radeon_get_bios`
>>>
>>> static bool radeon_load_bios(struct radeon_device *rdev)
>>> {
>>>          const struct firmware *fw;
>>>          if (request_firmware(&fw, "radeon.bios", rdev->dev) != 0)
>>>                  return false;
>>>
>>>          rdev->bios = kmalloc(fw->size, GFP_KERNEL);
>>>          if (!rdev->bios) {
>>>                  release_firmware(fw);
>>>                  DRM_ERROR("Unable to allocate bios\n");
>>>                  return false;
>>>          }
>>>
>>>          memcpy(rdev->bios, fw->data, fw->size);
>>>          release_firmware(fw);
>>>          return true;
>>> }
>>>
>>> I hope this helps anyone else trying find a way to just pass the bios
>>> into the driver without having to resort to hvm. It would be nice to
>>> see a patch to `drivers/pci/rom.c` that does the same thing, but
>>> chooses a bios name based on the pci device instead, allowing the
>>> opportunity to override the pci rom with any provided file both for
>>> testing/debugging and corner cases like this one here.
>> Thankyou for this - it is very interesting to see.
>>
>> Given your description of the ROM issue and how you fixed it, I presume
>> it should be easy to fix by allowing pci-front to map the Option ROM BAR
>> of an assigned device?  As far as I can tell, all you did was make the
>> contents of the Option ROM available by an alternative means.
>>
>> ~Andrew
>
> Correct, it was just providing the rom to the driver. Is there already
> a way to allow pci-front to pass this, or does it require a patch? I
> do not know the code base very well, nor do I have much experience
> with how PCI operates, I was pretty excited to get this far :)

I am not very familiar with pci-{front,back}, although lack of Option
ROM mapping seems like the obvious explanation.

Konrad - any ideas?

~Andrew