Re: PCI trouble on mvebu (Turris Omnia)

["Toke Høiland-Jørgensen" <toke@redhat.com>]

Bjorn Helgaas <helgaas@kernel.org> writes:

> Another experiment: build kernel without CONFIG_PCIEASPM, set $ROOT
> and $NIC appropriately, and try the following:
>
>   # Set $ROOT and $NIC (update to match your system):
>
>     # ROOT=00:02.0
>     # NIC=02:00.0

(these matched the ath10k card, so just went with that)

>   # Dump the Root Port and NIC Link registers:
>
>     # setpci -s$ROOT CAP_EXP+0xc.l              # Link Capabilities
>     # setpci -s$ROOT CAP_EXP+0x10.w             # Link Control
>     # setpci -s$ROOT CAP_EXP+0x12.w             # Link Status

# setpci -s$ROOT CAP_EXP+0xc.l
0003ac12
# setpci -s$ROOT CAP_EXP+0x10.w
0040
# setpci -s$ROOT CAP_EXP+0x12.w
1011

>     # setpci -s$NIC  CAP_EXP+0xc.l              # Link Capabilities
>     # setpci -s$NIC  CAP_EXP+0x10.w             # Link Control
>     # setpci -s$NIC  CAP_EXP+0x12.w             # Link Status

# setpci -s$NIC CAP_EXP+0xc.l
00036c11
# setpci -s$NIC CAP_EXP+0x10.w
0000
# setpci -s$NIC CAP_EXP+0x12.w
1011

>   # Retrain the link:
>
>     # setpci -s$ROOT CAP_EXP+0x10.w=0x0020      # Link Control Retrain Link
>     # sleep 1
>     # setpci -s$ROOT CAP_EXP+0x12.w             # Link Status
>     # setpci -s$NIC  CAP_EXP+0x12.w             # Link Status

# setpci -s$ROOT CAP_EXP+0x10.w=0x0020
# sleep 1
# setpci -s$ROOT CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0x12.w
setpci: 0000:02:00.0: Instance #0 of Capability 0010 not found - there are no capabilities with that id.
# setpci -s$ROOT CAP_EXP+0x10.w
0000

(nothing in the dmesg either) - rebooted before trying the below:

>   # Set CommClk+ and retrain the link:
>
>     # setpci -s$NIC  CAP_EXP+0x10.w=0x0040      # Link Control Common Clock
>     # setpci -s$ROOT CAP_EXP+0x10.w=0x0040      # Link Control Common Clock
>     # setpci -s$ROOT CAP_EXP+0x10.w=0x0060      # Link Control RL + CC
>     # sleep 1
>     # setpci -s$ROOT CAP_EXP+0x12.w             # Link Status
>     # setpci -s$NIC  CAP_EXP+0x12.w             # Link Status

# setpci -s$NIC CAP_EXP+0x10.w=0x0040
# setpci -s$ROOT CAP_EXP+0x10.w=0x0040
# setpci -s$ROOT CAP_EXP+0x10.w=0x0060
# sleep 1
# setpci -s$ROOT CAP_EXP+0x12.w
1011
# setpci -s$NIC CAP_EXP+0x12.w
setpci: 0000:02:00.0: Instance #0 of Capability 0010 not found - there are no capabilities with that id.

# lspci -v
00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 6820 (rev 04) (prog-if 00 [Normal decode])
        Device tree node: /sys/firmware/devicetree/base/soc/pcie/pcie@1,0
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: e0000000-e00fffff [size=1M]
        Prefetchable memory behind bridge: 00000000-000fffff [size=1M]
        Expansion ROM at e0100000 [virtual] [disabled] [size=2K]
        Capabilities: [40] Express Root Port (Slot+), MSI 00
lspci: Unable to load libkmod resources: error -12

00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 6820 (rev 04) (prog-if 00 [Normal decode])
        Device tree node: /sys/firmware/devicetree/base/soc/pcie/pcie@2,0
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: e0200000-e04fffff [size=3M]
        Prefetchable memory behind bridge: 00000000-000fffff [size=1M]
        Expansion ROM at e0500000 [virtual] [disabled] [size=2K]
        Capabilities: [40] Express Root Port (Slot+), MSI 00

00:03.0 PCI bridge: Marvell Technology Group Ltd. Device 6820 (rev 04) (prog-if 00 [Normal decode])
        Device tree node: /sys/firmware/devicetree/base/soc/pcie/pcie@3,0
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: e0600000-e07fffff [size=2M]
        Prefetchable memory behind bridge: 00000000-000fffff [size=1M]
        Expansion ROM at e0800000 [virtual] [disabled] [size=2K]
        Capabilities: [40] Express Root Port (Slot+), MSI 00

01:00.0 Network controller: Qualcomm Atheros AR9287 Wireless Network Adapter (PCI-Express) (rev 01)
        Subsystem: Qualcomm Atheros AR9287 Wireless Network Adapter (PCI-Express)
        Flags: bus master, fast devsel, latency 0, IRQ 60
        Memory at e0000000 (64-bit, non-prefetchable) [size=64K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
        Capabilities: [60] Express Legacy Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Capabilities: [160] Device Serial Number 00-15-17-ff-ff-24-14-12
        Capabilities: [170] Power Budgeting <?>
        Kernel driver in use: ath9k

02:00.0 Network controller: Qualcomm Atheros QCA986x/988x 802.11ac Wireless Network Adapter (rev ff) (prog-if ff)
        !!! Unknown header type 7f
        Kernel driver in use: ath10k_pci

03:00.0 Network controller: MEDIATEK Corp. Device 7612
        Subsystem: MEDIATEK Corp. Device 7612
        Flags: bus master, fast devsel, latency 0, IRQ 63
        Memory at e0600000 (64-bit, non-prefetchable) [size=1M]
        Expansion ROM at e0700000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [148] Device Serial Number 00-00-00-00-00-00-00-00
        Capabilities: [158] Latency Tolerance Reporting
        Capabilities: [160] L1 PM Substates
        Kernel driver in use: mt76x2e


-Toke