Multiple Qualcomm Hamilton cards on same host - none working

Multiple Qualcomm Hamilton cards on same host - none working

[david Atkins]

Hi,

We need to run multiple wcn7850 (Hamilton) cards on the same host. 

A single wcn7850 works fine, more than one and all fail

Logs from a working setup with a single card using Ubuntu 23.10.1 desktop install

[    2.494236] ath12k_pci 0000:03:00.0: BAR 0: assigned [mem 0xfe200000-0xfe3fffff 64bit]
[    2.494256] ath12k_pci 0000:03:00.0: enabling device (0000 -> 0002)
[    2.496200] ath12k_pci 0000:03:00.0: Hardware name: wcn7850 hw2.0
[    4.993821] ath12k_pci 0000:03:00.0: chip_id 0x2 chip_family 0x4 board_id 0xff soc_id 0x40170200
[    4.993827] ath12k_pci 0000:03:00.0: fw_version 0x100301e1 fw_build_timestamp 2023-12-06 04:05 fw_build_id QC_IMAGE_VERSION_STRING=WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
[    5.003142] ath12k_pci 0000:03:00.0: failed to fetch board data for bus=pci,qmi-chip-id=2,qmi-board-id=255 from ath12k/WCN7850/hw2.0/board-2.bin
[    5.196430] ath12k_pci 0000:03:00.0 wlp3s0: renamed from wlan0
[    9.446866] UBSAN: array-index-out-of-bounds in /build/linux-9czVP0/linux-6.5.0/drivers/net/wireless/ath/ath12k/mac.c:2774:17
[    9.447067]  ath12k_mac_op_hw_scan+0x39c/0x3e0 [ath12k]

Logs form a similar setup with 2 cards

sudo dmesg | grep ath12k

gives me

[    3.570029] ath12k_pci 0000:01:00.0: BAR 0: assigned [mem 0xfc800000-0xfc9fffff 64bit]
[    3.570046] ath12k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    3.570347] ath12k_pci 0000:01:00.0: Hardware name: wcn7850 hw2.0
[    3.659620] ath12k_pci 0000:05:00.0: BAR 0: assigned [mem 0xfc400000-0xfc5fffff 64bit]
[    3.659648] ath12k_pci 0000:05:00.0: enabling device (0000 -> 0002)
[    3.660023] ath12k_pci 0000:05:00.0: Hardware name: wcn7850 hw2.0
[    4.910790] ath12k_pci 0000:01:00.0: Host capability request failed, result: 1, err: 90
[    4.910795] ath12k_pci 0000:01:00.0: qmi failed to send host cap QMI:-22
[    4.912112] ath12k_pci 0000:05:00.0: Host capability request failed, result: 1, err: 90
[    4.912116] ath12k_pci 0000:05:00.0: qmi failed to send host cap QMI:-22
[    4.913414] ath12k_pci 0000:01:00.0: Host capability request failed, result: 1, err: 90
[    4.913418] ath12k_pci 0000:01:00.0: qmi failed to send host cap QMI:-22

After running

sudo dmesg -n 8

then run

sudo dmesg | grep ath12k

again 

[49523.559674] ath12k_pci 0000:01:00.0: BAR 0: assigned [mem 0xfc800000-0xfc9fffff 64bit]
[49523.559965] ath12k_pci 0000:01:00.0: Hardware name: wcn7850 hw2.0
[49523.615645] ath12k_pci 0000:05:00.0: BAR 0: assigned [mem 0xfc400000-0xfc5fffff 64bit]
[49523.616109] ath12k_pci 0000:05:00.0: Hardware name: wcn7850 hw2.0
[49524.094457] ath12k_pci 0000:05:00.0: Host capability request failed, result: 1, err: 90
[49524.094469] ath12k_pci 0000:05:00.0: qmi failed to send host cap QMI:-22
[49524.095445] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000000 flags=0x0000]
[49524.095668] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000000 flags=0x0020]
[49524.096223] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000000 flags=0x0000]
[49524.096307] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7540000 flags=0x0020]
[49524.096316] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000040 flags=0x0000]
[49524.096327] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000080 flags=0x0000]
[49524.096335] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb70000c0 flags=0x0000]
[49524.096343] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000100 flags=0x0000]
[49524.096353] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000140 flags=0x0000]
[49524.096361] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000180 flags=0x0000]
[49524.099587] ath12k_pci 0000:05:00.0: chip_id 0x2 chip_family 0x4 board_id 0x10e soc_id 0x40170200
[49524.099593] ath12k_pci 0000:05:00.0: fw_version 0x100301e1 fw_build_timestamp 2023-12-06 04:05 fw_build_id QC_IMAGE_VERSION_STRING=WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
[49524.110011] ath12k_pci 0000:05:00.0: failed to fetch board data for bus=pci,qmi-chip-id=2,qmi-board-id=270 from ath12k/WCN7850/hw2.0/board-2.bin
[49534.169968] ath12k_pci 0000:05:00.0: qmi failed set mode request, mode: 0, err = -110
[49534.169981] ath12k_pci 0000:05:00.0: qmi failed to send wlan fw mode:-110
[49534.169986] ath12k_pci 0000:05:00.0: failed to send firmware start: -110
[49534.169989] ath12k_pci 0000:05:00.0: failed to start firmware: -110

The error

[49524.094457] ath12k_pci 0000:05:00.0: Host capability request failed, result: 1, err: 90
[49524.094469] ath12k_pci 0000:05:00.0: qmi failed to send host cap QMI:-22

Would appear to originate in ath12k_qmi_host_cap_send(), I assume that all works but that the response read from qmi_txn_wait() is 90, which from qmi.h is QMI_ERR_INCOMPATIBLE_STATE_V01. No idea where to go on that!

Both the working and non working have this line

[    5.003142] ath12k_pci 0000:03:00.0: failed to fetch board data for bus=pci,qmi-chip-id=2,qmi-board-id=255 from ath12k/WCN7850/hw2.0/board-2.bin

I used firmware files from

https://github.com/kvalo/ath11k-firmware/tree/master/ath12k-testing

using the files from 

ath12k-testing/WCN7850/hw2.0/WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3

copied into the appropriate firmware file location

the last error

[49534.169968] ath12k_pci 0000:05:00.0: qmi failed set mode request, mode: 0, err = -110
[49534.169981] ath12k_pci 0000:05:00.0: qmi failed to send wlan fw mode:-110
[49534.169986] ath12k_pci 0000:05:00.0: failed to send firmware start: -110
[49534.169989] ath12k_pci 0000:05:00.0: failed to start firmware: -110

Appears to originate in ath12k_qmi_wlanfw_mode_send(), not able to find a meaning for the -110 code.

Not sure if the later error is relevant given the first has occurred though!

Any ideas?

David Atkins

Re: Multiple Qualcomm Hamilton cards on same host - none working

[Kalle Valo]

david Atkins <david@3adesign.co.uk> writes:

> We need to run multiple wcn7850 (Hamilton) cards on the same host.
>
> A single wcn7850 works fine, more than one and all fail

Supporting multiple WCN7850 boards is a firmware limitation (and we need
some extra ath12k patches). If you can please submit a report to
bugzilla, easier to track that way:

https://wireless.wiki.kernel.org/en/users/drivers/ath12k/bugreport

> Logs from a working setup with a single card using Ubuntu 23.10.1 desktop install

BTW we don't support distro kernels as we don't know what patches they
have. When reporting bugs to us please use vanilla kernel.org releases
from:

https://www.kernel.org/

And even better if you can use the latest and greatest from ath.git
master branch:

https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/

And do include the output from 'git describe --tags' when testing
kernels so that we can exactly identify the release.

-- 
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

RE: Multiple Qualcomm Hamilton cards on same host - none working

[david Atkins]

Hi,

I've pulled and built, is this a good place to report issues from (if still extant!)

Linux openwrt-wifi-test 6.8.0-rc3-wt-ath+ #1 SMP PREEMPT_DYNAMIC Thu Feb  8 07:30:15 GMT 2024 x86_64 x86_64 x86_64 GNU/Linux

David

-----Original Message-----
From: Kalle Valo <kvalo@kernel.org> 
Sent: Wednesday, February 7, 2024 11:34 AM
To: david Atkins <david@3adesign.co.uk>
Cc: ath12k@lists.infradead.org
Subject: Re: Multiple Qualcomm Hamilton cards on same host - none working

david Atkins <david@3adesign.co.uk> writes:

> We need to run multiple wcn7850 (Hamilton) cards on the same host.
>
> A single wcn7850 works fine, more than one and all fail

Supporting multiple WCN7850 boards is a firmware limitation (and we need some extra ath12k patches). If you can please submit a report to bugzilla, easier to track that way:

https://wireless.wiki.kernel.org/en/users/drivers/ath12k/bugreport

> Logs from a working setup with a single card using Ubuntu 23.10.1 
> desktop install

BTW we don't support distro kernels as we don't know what patches they have. When reporting bugs to us please use vanilla kernel.org releases
from:

https://www.kernel.org/

And even better if you can use the latest and greatest from ath.git master branch:

https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/

And do include the output from 'git describe --tags' when testing kernels so that we can exactly identify the release.

--
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

RE: Multiple Qualcomm Hamilton cards on same host - none working

[david Atkins]

Done as requested!


-----Original Message-----
From: Kalle Valo <kvalo@kernel.org> 
Sent: Wednesday, February 7, 2024 11:34 AM
To: david Atkins <david@3adesign.co.uk>
Cc: ath12k@lists.infradead.org
Subject: Re: Multiple Qualcomm Hamilton cards on same host - none working

david Atkins <david@3adesign.co.uk> writes:

> We need to run multiple wcn7850 (Hamilton) cards on the same host.
>
> A single wcn7850 works fine, more than one and all fail

Supporting multiple WCN7850 boards is a firmware limitation (and we need some extra ath12k patches). If you can please submit a report to bugzilla, easier to track that way:

https://wireless.wiki.kernel.org/en/users/drivers/ath12k/bugreport

> Logs from a working setup with a single card using Ubuntu 23.10.1 
> desktop install

BTW we don't support distro kernels as we don't know what patches they have. When reporting bugs to us please use vanilla kernel.org releases
from:

https://www.kernel.org/

And even better if you can use the latest and greatest from ath.git master branch:

https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/

And do include the output from 'git describe --tags' when testing kernels so that we can exactly identify the release.

--
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

RE: Multiple Qualcomm Hamilton cards on same host - none working

[david Atkins]

Hi,

Update on this, with an Intel CPU the first NIC is found but the second fails so slightly different behaviour, I'll update bug registered with logs

I note there was a patch with a comment:

" * support having multiple identical PCI devices (firmware needs to
  have ATH12K_FW_FEATURE_MULTI_QRTR_ID) "

would this address the issue? and is the updated firmware available?

In discussion with Qualcomm this week I think understood from them that the ath12k driver will not support Wi-Fi 7 in its current form, is this correct?

David

-----Original Message-----
From: Kalle Valo <kvalo@kernel.org> 
Sent: Wednesday, February 7, 2024 11:34 AM
To: david Atkins <david@3adesign.co.uk>
Cc: ath12k@lists.infradead.org
Subject: Re: Multiple Qualcomm Hamilton cards on same host - none working

david Atkins <david@3adesign.co.uk> writes:

> We need to run multiple wcn7850 (Hamilton) cards on the same host.
>
> A single wcn7850 works fine, more than one and all fail

Supporting multiple WCN7850 boards is a firmware limitation (and we need some extra ath12k patches). If you can please submit a report to bugzilla, easier to track that way:

https://wireless.wiki.kernel.org/en/users/drivers/ath12k/bugreport

> Logs from a working setup with a single card using Ubuntu 23.10.1 
> desktop install

BTW we don't support distro kernels as we don't know what patches they have. When reporting bugs to us please use vanilla kernel.org releases
from:

https://www.kernel.org/

And even better if you can use the latest and greatest from ath.git master branch:

https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/

And do include the output from 'git describe --tags' when testing kernels so that we can exactly identify the release.

--
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

Re: Multiple Qualcomm Hamilton cards on same host - none working

[Jeff Johnson]

On 2/28/2024 11:21 PM, david Atkins wrote:
> Hi,
> 
> Update on this, with an Intel CPU the first NIC is found but the second fails so slightly different behaviour, I'll update bug registered with logs

FYI on Linux Kernel mailing lists it is preferred to either have
interleaved or bottom-posted responses.
<https://docs.kernel.org/process/submitting-patches.html#use-trimmed-interleaved-replies-in-email-discussions>

> 
> I note there was a patch with a comment:
> 
> " * support having multiple identical PCI devices (firmware needs to
>   have ATH12K_FW_FEATURE_MULTI_QRTR_ID) "
> 
> would this address the issue? and is the updated firmware available?

New firmware would be required. Kalle and I are actively working with
the firmware team to get updated firmware posted publicly.

> In discussion with Qualcomm this week I think understood from them that the ath12k driver will not support Wi-Fi 7 in its current form, is this correct?

The current driver has support for some of the WiFi 7 features but the
major addition, multi-link operation, is not currently available.