[5.17 regression] "x86/PCI: Ignore E820 reservations for bridge windows on newer systems" breaks suspend/resume

[Hans de Goede <hdegoede@redhat.com>]

Hi All,

Unfortunately I've just learned that commit 7f7b4236f204 ("x86/PCI:
Ignore E820 reservations for bridge windows on newer systems"):

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7f7b4236f2040d19df1ddaf30047128b41e78de7

breaks suspend/resume on at least one laptop model, the Lenovo ThinkPad
X1 gen 2, see:
https://bugzilla.redhat.com/show_bug.cgi?id=2029207

This regression was actually caught be Fedora already carrying this
patch for a while now and as such it has been reproduced with 5.15
with an older version of the patch which still allowed turning the
new behavior of by adding "pci=use_e820". Dmesg output with and
without the option has just been attached to the bug, I've not
analyzed this any further yet.

I guess that for now this means that we need to revert commit
7f7b4236f204. Rafael, I'll send you a revert with a commit msg
explaining why this needs to be reverted tomorrow.

More interesting IMHO is finding out another solution. Both the touchpad
problem which got me looking into this:
https://bugzilla.redhat.com/show_bug.cgi?id=1868899

As well as the thunderbolt hotplug issue Mika was looking at:
https://bugzilla.kernel.org/show_bug.cgi?id=206459

both are cases where we fail to find a memory-window for a
BAR which has not been setup yet.

So I see a couple of options here:

1. Detect that the e820 reservations fully cover (one of)
the PCI bridge main 32 bit memory windows and if that happens
ignore them. This actually was my first plan when I started
working on this. In the end I choose the other option
because Bjorn indicated that in hindsight honoring the e820
reservations might have been a mistake and maybe we should
get rid of honoring them all together.

2. Have a flag which, when we fail to alloc a 32 bit
(or 64 bit) memory PCI BAR, is set if not already set
and then retry the alloc. And make the e820 reservation
carve-out get skipped in this case.

3. When booting with pci=nocrs as a workaround for
the touchpad case a 64 but memory window ends up getting
used. There already is some special handling for some
AMD bridges where if there are no 64 bit memory Windows
in the _CRS for the bridge, one gets added. Maybe we need
to do the same for Intel bridges ?

Please let me know which of these options you think I should
try to implement next; of course alternative ideas for fixing
this are also welcome.

Regards,

Hans