Re: How to triage an "Invalid Key" error message?

[James Prestwood <prestwoj@gmail.com>]

Hi Christian,

On 1/20/25 7:36 AM, Christian Hewitt wrote:
>> On 18 Jan 2025, at 8:38 pm, KeithG <ys3al35l@gmail.com> wrote:
>>
>> Christian,
>>
>> Is this, perhaps, on devices with the brcmfmac chip? I have seen this
>> recently with on my Pis with wpa2 and wpa3 APs.
> The majority of reports are from RPi 4/5 users which have brcmfmac
> chips, but 85% of our distro installed base runs an RPi board so
> that mostly reflects our demographics. I’ve also seen reports on
> Amlogic and Rockchip SoC devices (also using brcmfmac); but to
> offset theories on it being brcmfmac specific, we’ve also had
> reports from users with Intel iwlwifi hardware and someone using
> an RTL8723BS chip. So it’s more likely to be an interaction with
> common code not driver code.
>
> At one point we suspected an issue with WPA3, but that was triaged
> with RPi developers and we now disable SAE offload which allows
> boards to connect to mixed WPA2/WPA3 networks, see:
>
> https://forum.libreelec.tv/thread/29155-rpi5-network-disappeared
> https://github.com/LibreELEC/LibreELEC.tv/pull/9477/files
>
> In the following forum and GitHub issue threads. There’s lots of
> assumptions, guesswork, and false conclusions, but among the noise
> there are some snippets of iwd debug log:
>
> https://github.com/LibreELEC/LibreELEC.tv/issues/7166

This one actually has some iwmon logs. Looks like the kernel is sending 
a failure within ~100ms of the connection which is odd. IWD waits for a 
successful connect event before starting the 4-way handshake so its not 
even getting to that point:

< Request: Connect (0x2e) len 172 [ack]                                
5.644853
...
 > Event: Connect (0x2e) len 36                                         
5.735838
     Wiphy: 0 (0x00000000)
     Interface Index: 3 (0x00000003)
     MAC Address 00:00:00:00:00:00
     Status Code: 16 (0x0010)

There is also a core dump attached, but the back trace isn't of much use.

> https://github.com/LibreELEC/LibreELEC.tv/issues/8731

This issue _appears_ as if wpa_supplicant is also running along side 
IWD. That is generally why you see this log:

Mar 16 21:10:35 pineh64-b iwd[1046]: Unexpected connection related event 
-- is another supplicant running?

> https://github.com/LibreELEC/LibreELEC.tv/issues/8760
This seems the same as #7166.
> https://github.com/LibreELEC/LibreELEC.tv/issues/8762
Not a whole lot in here, but apparently resolved.
>
> I’ll need to rig a distro image for greater debug (as per James’
> instructions) to see if we can obtain more log output.
>
> Christian
>
So overall the "invalid-key" issue seems to be due to brcmfmac 
signalling a failure very early on. In a perfect world it would be great 
if we could capture debug/iwmon logs of both IWD trying to connect and 
wpa_supplicant, so we can compare the iwmon logs and see what each 
supplicant is sending to the kernel. The messages _looked_ ok from IWD 
but if wpa_supplicant works fine there is obviously a difference 
somewhere. As Keith knows, brcmfmac is kinda a pain to support as its 
the odd-ball as far as drivers go. IWD doesn't get much contributions 
from wifi vendors so any odd quirks only get solved by trial and error.

Thanks,

James