Re: Question about supporting AMD eGPU hot plug case

[Andrey Grodzovsky <andrey.grodzovsky@amd.com>]

Ping

Andrey

On 2021-03-12 10:34 a.m., Andrey Grodzovsky wrote:
>
>
> On 2021-03-12 4:03 a.m., Christian König wrote:
>> Am 11.03.21 um 23:40 schrieb Andrey Grodzovsky:
>>> [SNIP]
>>>>> The expected result is they all move closer to the start of PCI 
>>>>> address
>>>>> space.
>>>>>
>>>>
>>>> Ok, I updated as you described. Also I removed PCI conf command to 
>>>> stop
>>>> address decoding and restart later as I noticed PCI core does it 
>>>> itself
>>>> when needed.
>>>> I tested now also with graphic desktop enabled while submitting
>>>> 3d draw commands and seems like under this scenario everything still
>>>> works. Again, this all needs to be tested with VRAM BAR move as then
>>>> I believe I will see more issues like handling of MMIO mapped VRAM 
>>>> objects (like GART table). In case you do have an AMD card you 
>>>> could also maybe give it a try. In the meanwhile I will add support 
>>>> to ioremapping of those VRAM objects.
>>>>
>>>> Andrey
>>>
>>> Just an update, added support for unmaping/remapping of all VRAM
>>> objects, both user space mmaped and kernel ioremaped. Seems to work
>>> ok but again, without forcing VRAM BAR to move I can't be sure.
>>> Alex, Chsristian - take a look when you have some time to give me some
>>> initial feedback on the amdgpu side.
>>>
>>> The code is at 
>>> https://cgit.freedesktop.org/~agrodzov/linux/log/?h=yadro%2Fpcie_hotplug%2Fmovable_bars_v9.1 
>>>
>>
>> Mhm, that let's userspace busy retry until the BAR movement is done.
>>
>> Not sure if that can't live lock somehow.
>>
>> Christian.
>
> In my testing it didn't but, I can instead route them to some
> global static dummy page while BARs are moving and then when everything
> done just invalidate the device address space again and let the
> pagefaults fill in valid PFNs again.
>
> Andrey
>
>>
>>>
>>> Andrey
>>