[Intel-wired-lan] PCI Advanced Features (especially FLR) missing in several newer Intel integrated ethernet devices

["Marek Marczykowski-Górecki" <marmarek@invisiblethingslab.com>]

[-- Attachment #1: Type: text/plain, Size: 2717 bytes --]

Hi,

I've got already at least 3 reports about e1000e device that is missing
FLR (which makes it harder to passthrough to a VM, among other things).
Interestingly, when looking at config space, the capability appears to
be there, just isn't linked to the list. For example:

    sudo lspci -nnvxxxs 00:1f.6
    00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:57a1]
        Subsystem: Lenovo Device [17aa:512b]
        Flags: bus master, fast devsel, latency 0, IRQ 19
        Memory at ab700000 (32-bit, non-prefetchable) [size=128K]
        Capabilities: [c8] Power Management version 3
        Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Kernel driver in use: pciback
        Kernel modules: e1000e
    00: 86 80 a1 57 06 04 10 00 00 00 00 02 00 00 00 00
    10: 00 00 70 ab 00 00 00 00 00 00 00 00 00 00 00 00
    20: 00 00 00 00 00 00 00 00 00 00 00 00 aa 17 2b 51
    30: 00 00 00 00 c8 00 00 00 00 00 00 00 ff 04 00 00
    40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    80: 28 00 00 00 08 00 00 00 00 00 00 00 00 00 00 00
    90: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
    a0: ff ff ff ff ff ff ff ff 03 10 03 10 00 00 00 00
    b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    c0: 00 00 00 00 00 00 00 00 01 d0 23 c8 08 20 00 00
    d0: 05 00 81 00 b8 12 e0 fe 00 00 00 00 00 00 00 00
    e0: 13 00 06 03 00 00 00 00 00 00 00 00 00 00 00 00
    f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

As you can see, the "PCI Advanced Features" is not listed. But, if you
traverse the list manually, you can see that the capability is there at
0xe0 - it just isn't linked at 0xd1 (0xd1 has 0x00 instead of 0xe0). And
it appears it isn't just coincidence:
 - 0x03 at 0xe3 says FLR is supported (bit FLR_CAP is set)
 - writing 0x01 to 0xe4 (INITIATE_FLR) does appear to reset the device

It's even possible to link the capability manually (write 0xe0 to 0xd1)
and then Linux is happy to use it via the standard `echo 1 >
/sys/devices/.../reset` interface (but, ofc, such linking doesn't
survive the reset).

Based on reports I've received, it applies at least to devices with the
following VID:DID:
- 8086:550a
- 8086:57a1
- 8086:57a0

Some of those reports at https://github.com/QubesOS/qubes-issues/issues/10657

My question is: why the capability is not linked? Does it want a fix in
firmware? Or maybe a quirk in the kernel to restore working FLR?

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]