brcmfmac AP mode

brcmfmac AP mode

[KeithG]

I have been trying to use teh brcmfmac device on my RPi to run in AP
mode to allow device configuration of wifi and such when ethernet is
not available. I am using connman and iwd to manage my internet
connections and have noticed that when the AP is running, the AP will
reset when iwd scans.

I set it up to be an AP so I can configure the network on the RPi via
a web interface. I first set up a as a separate device as an AP for
wlan0:

iw dev wlan0 interface add ap0 type __ap

I then give it a MAC different from the wlan0 so iwd can keep track of
which is which and the ssids will show up on wlan0 when I run 'iwctl
station wlan0 scan'

ip link set dev ap0 address b8:27:eb:00:00:00

I then start an AP with either iwd or hostapd/dnsmasq.

iwctl device ap0 set-property Mode ap
iwctl ap ap0 start-profile testAP

It will start the AP and then I can connect to the Pi from my laptop.
I am using iwd on wlan0 to scan for available SSIDs. Whenever iwd
scans, either manually from cli or automatically, it will reset the AP
and kick everyone off. It does this even when I use hostapd/dnsmasq to
control the AP. When I disable iwd and use wpa_supplicant for
scanning, though, I can set up the AP with hostapd and scan many times
via connmanctl and it will not disconnect.

Why does it kick everyone off when iwd scans? Why is iot more stable
with wpa_supplicant? Is there something I could/should do to make this
more stable? When I try this on an intel card, it is rock steady and I
have not seen a disconnect. I have tried the latest brcmfmac
(cycmfmac) firmware and see the same result.

https://github.com/Infineon/ifx-linux-firmware/tree/master/firmware

I am running RPiOS Bookworm and am fully up to date though it uses an
older firmware
[   11.513277] brcmfmac: F1 signature read @0x18000000=0x1542a9a6
[   11.533628] brcmfmac: brcmf_fw_alloc_request: using
brcm/brcmfmac43430b0-sdio for chip BCM43430/2
[   11.740252] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/2
wl0: Mar 31 2022 17:24:51 version 9.88.4.77 (g58bc5cc) FWID
01-3b307371

Is there any hope of stability when used like this?

Keith

Re: brcmfmac AP mode

[James Prestwood]

All,

On 2/12/24 5:36 PM, KeithG wrote:
> I have been trying to use teh brcmfmac device on my RPi to run in AP
> mode to allow device configuration of wifi and such when ethernet is
> not available. I am using connman and iwd to manage my internet
> connections and have noticed that when the AP is running, the AP will
> reset when iwd scans.
>
> I set it up to be an AP so I can configure the network on the RPi via
> a web interface. I first set up a as a separate device as an AP for
> wlan0:
>
> iw dev wlan0 interface add ap0 type __ap
>
> I then give it a MAC different from the wlan0 so iwd can keep track of
> which is which and the ssids will show up on wlan0 when I run 'iwctl
> station wlan0 scan'
>
> ip link set dev ap0 address b8:27:eb:00:00:00
>
> I then start an AP with either iwd or hostapd/dnsmasq.
>
> iwctl device ap0 set-property Mode ap
> iwctl ap ap0 start-profile testAP
>
> It will start the AP and then I can connect to the Pi from my laptop.
> I am using iwd on wlan0 to scan for available SSIDs. Whenever iwd
> scans, either manually from cli or automatically, it will reset the AP
> and kick everyone off. It does this even when I use hostapd/dnsmasq to
> control the AP. When I disable iwd and use wpa_supplicant for
> scanning, though, I can set up the AP with hostapd and scan many times
> via connmanctl and it will not disconnect.
>
> Why does it kick everyone off when iwd scans? Why is iot more stable
> with wpa_supplicant? Is there something I could/should do to make this
> more stable? When I try this on an intel card, it is rock steady and I
> have not seen a disconnect. I have tried the latest brcmfmac
> (cycmfmac) firmware and see the same result.
>
> https://github.com/Infineon/ifx-linux-firmware/tree/master/firmware
>
> I am running RPiOS Bookworm and am fully up to date though it uses an
> older firmware
> [   11.513277] brcmfmac: F1 signature read @0x18000000=0x1542a9a6
> [   11.533628] brcmfmac: brcmf_fw_alloc_request: using
> brcm/brcmfmac43430b0-sdio for chip BCM43430/2
> [   11.740252] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/2
> wl0: Mar 31 2022 17:24:51 version 9.88.4.77 (g58bc5cc) FWID
> 01-3b307371
>
> Is there any hope of stability when used like this?
>
> Keith
>
I was helping Keith out here and wanted to provide a bit more 
information. I found there were a few differences between IWD and 
wpa_supplicant related to scanning which may aid in figuring out why 
brcmfmac is behaving this way:

  - IWD scans using the wdev ID where wpa_supplicant uses ifindex. Not 
sure if this has anything to do with the difference behavior.

  - Passive scans (which IWD prefers) seem to exacerbate the behavior. 
Simple testing using "wpa_cli scan" showed wpa_supplicant was only using 
active scans. I also tested with iw and saw repeatable disconnects when 
passive scanning. Disconnects while using active scans (using iw) were 
much less frequent.

  - Scanning with IWD, wpa_supplicant, or iw, passive or active, always 
resulted in beacon loss for clients connected to the AP. This was 100% 
guaranteed. The clients just could recover when active scans were used 
over passive. But either way this does not seem like normal behavior, 
the AP interface should still be beaconing on its active channel during 
a scan right? If this isn't possible or can't be done reliably then 
should the interface combinations be changed to disallow concurrent sta 
+ AP mode interfaces?

Thanks,

James

Re: brcmfmac AP mode

[Johannes Berg]

I have no experience with brcmfmac, but ...

> 
> I was helping Keith out here and wanted to provide a bit more 
> information. I found there were a few differences between IWD and 
> wpa_supplicant related to scanning which may aid in figuring out why 
> brcmfmac is behaving this way:
> 
>   - IWD scans using the wdev ID where wpa_supplicant uses ifindex. Not 
> sure if this has anything to do with the difference behavior.

This is not even visible to the driver, it's entirely resolved in
nl80211, so no impact here.

>   - Passive scans (which IWD prefers) seem to exacerbate the behavior. 
> Simple testing using "wpa_cli scan" showed wpa_supplicant was only using 
> active scans. I also tested with iw and saw repeatable disconnects when 
> passive scanning. Disconnects while using active scans (using iw) were 
> much less frequent.

This makes sense, especially if it's __ap rather than __p2p_go type,
since it *has* to be off the channel for some time -- especially for
passive scans it has to be off-channel for more than a typical interval
to even do scanning correctly.

>   - Scanning with IWD, wpa_supplicant, or iw, passive or active, always 
> resulted in beacon loss for clients connected to the AP. This was 100% 
> guaranteed. The clients just could recover when active scans were used 
> over passive. But either way this does not seem like normal behavior, 
> the AP interface should still be beaconing on its active channel during 
> a scan right?

That's "normal" in the sense that you have to be off the channel for
scanning, and if you're off the channel you can't transmit beacons on
it?

For P2P GO rather than AP it should publish NoA descriptors in the
beacon to let clients know there won't be a beacon.

Now it's perhaps possible to time - especially active - scanning so you
can still beacon somewhat and inbetween, but I suppose the firmware
doesn't do that here.

In fact even outside of the beaconing, APs aren't expected to be off-
channel, clients can send data to them after all. Again P2P GO solves
that with NoA, but the spec itself has no good way to solve this and I'm
not even sure it would even want to.

In any case, you could argue that starting AP and client and then
scanning is pretty much _asking_ for trouble.

> If this isn't possible or can't be done reliably then 
> should the interface combinations be changed to disallow concurrent sta 
> + AP mode interfaces?

Maybe it could restrict it to P2P GO instead of AP? But then people will
anyway just notice that they can use P2P GO and connect arbitrary
clients to it (not just P2P client), then those clients won't honour the
NoA because they're not P2P, and then you're back to the exact same
situation...

johannes

Re: brcmfmac AP mode

[James Prestwood]

Hi Johannes,

On 2/14/24 6:34 AM, Johannes Berg wrote:
> I have no experience with brcmfmac, but ...
>
>> I was helping Keith out here and wanted to provide a bit more
>> information. I found there were a few differences between IWD and
>> wpa_supplicant related to scanning which may aid in figuring out why
>> brcmfmac is behaving this way:
>>
>>    - IWD scans using the wdev ID where wpa_supplicant uses ifindex. Not
>> sure if this has anything to do with the difference behavior.
> This is not even visible to the driver, it's entirely resolved in
> nl80211, so no impact here.
>
>>    - Passive scans (which IWD prefers) seem to exacerbate the behavior.
>> Simple testing using "wpa_cli scan" showed wpa_supplicant was only using
>> active scans. I also tested with iw and saw repeatable disconnects when
>> passive scanning. Disconnects while using active scans (using iw) were
>> much less frequent.
> This makes sense, especially if it's __ap rather than __p2p_go type,
> since it *has* to be off the channel for some time -- especially for
> passive scans it has to be off-channel for more than a typical interval
> to even do scanning correctly.
>
>>    - Scanning with IWD, wpa_supplicant, or iw, passive or active, always
>> resulted in beacon loss for clients connected to the AP. This was 100%
>> guaranteed. The clients just could recover when active scans were used
>> over passive. But either way this does not seem like normal behavior,
>> the AP interface should still be beaconing on its active channel during
>> a scan right?
> That's "normal" in the sense that you have to be off the channel for
> scanning, and if you're off the channel you can't transmit beacons on
> it?
>
> For P2P GO rather than AP it should publish NoA descriptors in the
> beacon to let clients know there won't be a beacon.
>
> Now it's perhaps possible to time - especially active - scanning so you
> can still beacon somewhat and inbetween, but I suppose the firmware
> doesn't do that here.
>
> In fact even outside of the beaconing, APs aren't expected to be off-
> channel, clients can send data to them after all. Again P2P GO solves
> that with NoA, but the spec itself has no good way to solve this and I'm
> not even sure it would even want to.
>
> In any case, you could argue that starting AP and client and then
> scanning is pretty much _asking_ for trouble.

Yes I suspected as much. It seems some firmware is just better at this 
than others. There is one use case there that I believe Kieth is 
targeting and that is new device onboarding which I'm sure your familiar 
with as just about every "smart" device uses it. Where the new devices 
starts an AP and your phone/App connects and provides credentials to the 
"real" network. The tricky part is having the new device scan for 
available networks while it has a client connected. Some drivers support 
AP scanning which maybe is really what should be used for this? Maybe 
that is optimized to actually work.

I guess I'll also ask, what _is_ the target use case for STA + AP 
interfaces running concurrently? If scanning is unreliable then 
connecting would also be most likely? so what can you actually do here?

>> If this isn't possible or can't be done reliably then
>> should the interface combinations be changed to disallow concurrent sta
>> + AP mode interfaces?
> Maybe it could restrict it to P2P GO instead of AP? But then people will
> anyway just notice that they can use P2P GO and connect arbitrary
> clients to it (not just P2P client), then those clients won't honour the
> NoA because they're not P2P, and then you're back to the exact same
> situation...
> johannes

Re: brcmfmac AP mode

[Johannes Berg]

> > In any case, you could argue that starting AP and client and then
> > scanning is pretty much _asking_ for trouble.
> 
> Yes I suspected as much. It seems some firmware is just better at this 
> than others.

Sure, that makes sense. Beacons are important, you could schedule (non-
passive) scans between them, and ... yeah you're going to lose frames,
but perhaps not the connection as likely.

> There is one use case there that I believe Kieth is 
> targeting and that is new device onboarding which I'm sure your familiar 
> with as just about every "smart" device uses it. Where the new devices 
> starts an AP and your phone/App connects and provides credentials to the 
> "real" network. 

:)
I always fight with that because typically it wants to transfer the
network credentials from my phone (and be on the same network), but many
devices I don't want to be on the same network ;-)

> The tricky part is having the new device scan for 
> available networks while it has a client connected. Some drivers support 
> AP scanning which maybe is really what should be used for this? Maybe 
> that is optimized to actually work.

That really depends how it's implemented, I wouldn't necessarily say
that "AP scanning" is really better. It's still a crutch.

> I guess I'll also ask, what _is_ the target use case for STA + AP 
> interfaces running concurrently? If scanning is unreliable then 
> connecting would also be most likely? so what can you actually do here?

You don't _have_ to be scanning on the station all the time if all you
care about is being connected to some other AP? But yeah, it's not all
_that_ useful to have that combination ... and yet I guess people want
it and want to live with the limitations. Especially if firmware is
better than what seems to be the case here.

I dunno. I don't think we support this combination for our devices, but
then again we only really support very limited AP mode.

johannes

Re: brcmfmac AP mode

[Nicolas Escande]

On Wed Feb 14, 2024 at 8:31 PM CET, James Prestwood wrote:

[...]

> I guess I'll also ask, what _is_ the target use case for STA + AP 
> interfaces running concurrently? If scanning is unreliable then 
> connecting would also be most likely? so what can you actually do here?

One very common use case of AP + STA is the multi AP setup. EasyMesh works that
way from what I recall. In this setup, you connect first with the STA then
create an AP on the same channel, so no scan involved when both AP & STA are up.