bcmdhd: Concurrent use of wlan0 and virtual interface in managed mode

bcmdhd: Concurrent use of wlan0 and virtual interface in managed mode

[Gucea Doru]

Hello, community

I'm a Master student doing my research on a subject related to Android
and multiple paths between two nodes. What I try to do now is to allow
a client to connect to multiple APs (Access Points).

I started with the simplest use-case where I have only two APs (AP1
and AP2) and an Android client  - a Nexus 5 device running the Kernel
from [1]. For the connection with AP1 I used the already present
interface wlan0. For the connection with AP2 I tried to use a newly
created virtual interface but the wpa_supplicant connection is not
successful. I'm not sure if this is a driver limitation or a driver
bug.

I will detail what I've done. After killing the wpa_supplicant process
I followed the steps:
--- Created a virtual interface in the station mode ---
root@hammerhead:/ # iw phy phy0 interface add vif_AP2 type station
root@hammerhead:/ # ip link set dev vif_AP2 address 64:89:9a:81:0f:22
root@hammerhead:/ # ip link set dev vif_AP2 up

---- Connection to AP1 - SUCCESS ----
root@hammerhead:/data # wpa_supplicant -iwlan0 -c/data/tmp/preshared_key_AP1
The dmesg log from [2] proved that the connection was successful.

---- Connection to AP2 - FAILED ----
root@hammerhead:/ # wpa_supplicant -ivif_AP2 -c/data/tmp/preshared_key_AP2
The dmesg log from [3] proved that the connection failed.

It seems that the connection error is due to a failed PNO setup [3].
Also, the failed PNO setup seems to prevent the AP scanning process
from running. More accurate, what is this PNO setup? I looked into the
Kernel code [4] but I can't figure it out how I could avoid this
error.

[1] https://github.com/gdetal/mptcp_nexus5
[2] https://drive.google.com/file/d/0B5SBH08PU_ChYzNIV3VxUWlqWGs/view?usp=sharing
[3] https://drive.google.com/file/d/0B5SBH08PU_ChNHdOck5UbjNERjQ/view?usp=sharing
[4] https://github.com/gdetal/mptcp_nexus5/blob/master/drivers/net/wireless/bcmdhd/dhd_pno.c

Any help is greatly appreciated,
Doru

Re: bcmdhd: Concurrent use of wlan0 and virtual interface in managed mode

[Arend van Spriel]

On 08/23/2015 05:15 PM, Gucea Doru wrote:
> Hello, community

As you are using bcmdhd I don't think this particular community cares a 
lot about it. But I will take the bait ;-)

> I'm a Master student doing my research on a subject related to Android
> and multiple paths between two nodes. What I try to do now is to allow
> a client to connect to multiple APs (Access Points).
>
> I started with the simplest use-case where I have only two APs (AP1
> and AP2) and an Android client  - a Nexus 5 device running the Kernel
> from [1]. For the connection with AP1 I used the already present
> interface wlan0. For the connection with AP2 I tried to use a newly
> created virtual interface but the wpa_supplicant connection is not
> successful. I'm not sure if this is a driver limitation or a driver
> bug.

I am not sure either. For the upstream brcmfmac we don't allow it at the 
moment. I am kinda surprised you were able to create the additional 
interface. There are a couple of things that I can think of here which 
may cause issues:

- APs operating on different channels.
- Both STA interfaces have powersave enabled.
- Firmware/driver do not support multi STA.

If APs are operating on the same channel it may work, but you probably 
won't have advantage of MP-TCP because of medium use.

Regards,
Arend

> I will detail what I've done. After killing the wpa_supplicant process
> I followed the steps:
> --- Created a virtual interface in the station mode ---
> root@hammerhead:/ # iw phy phy0 interface add vif_AP2 type station
> root@hammerhead:/ # ip link set dev vif_AP2 address 64:89:9a:81:0f:22
> root@hammerhead:/ # ip link set dev vif_AP2 up
>
> ---- Connection to AP1 - SUCCESS ----
> root@hammerhead:/data # wpa_supplicant -iwlan0 -c/data/tmp/preshared_key_AP1
> The dmesg log from [2] proved that the connection was successful.
>
> ---- Connection to AP2 - FAILED ----
> root@hammerhead:/ # wpa_supplicant -ivif_AP2 -c/data/tmp/preshared_key_AP2
> The dmesg log from [3] proved that the connection failed.
>
> It seems that the connection error is due to a failed PNO setup [3].
> Also, the failed PNO setup seems to prevent the AP scanning process
> from running. More accurate, what is this PNO setup? I looked into the
> Kernel code [4] but I can't figure it out how I could avoid this
> error.

PNO is preferred network offload in which firmware is configured with a 
number of ssids and it will notify host when it finds one of those.

wpa_supplicant will do a couple of regular scans after which it requests 
a scheduled_scan which the driver will use to configure PNO in firmware. 
Looking at the log in [3] I get the impression this multi-STA thing is 
not working at all. The regular scans don't find any AP, which is 
already suspect. The PNO setup is rejected by the driver, because the 
other interface is already connected to AP1.

Regards,
Arend

> [1] https://github.com/gdetal/mptcp_nexus5
> [2] https://drive.google.com/file/d/0B5SBH08PU_ChYzNIV3VxUWlqWGs/view?usp=sharing
> [3] https://drive.google.com/file/d/0B5SBH08PU_ChNHdOck5UbjNERjQ/view?usp=sharing
> [4] https://github.com/gdetal/mptcp_nexus5/blob/master/drivers/net/wireless/bcmdhd/dhd_pno.c
>
> Any help is greatly appreciated,
> Doru
>

Re: bcmdhd: Concurrent use of wlan0 and virtual interface in managed mode

[Gucea Doru]

Hello, Arend

On Mon, Aug 24, 2015 at 11:08 AM, Arend van Spriel <arend@broadcom.com> wrote:
> On 08/23/2015 05:15 PM, Gucea Doru wrote:
>>
>> Hello, community
>
>
> As you are using bcmdhd I don't think this particular community cares a lot
> about it. But I will take the bait ;-)
>

Thanks for taking the bait :). Should we move this discussion to a
bcmdhd specific mail-list?

>> I started with the simplest use-case where I have only two APs (AP1
>> and AP2) and an Android client  - a Nexus 5 device running the Kernel
>> from [1]. For the connection with AP1 I used the already present
>> interface wlan0. For the connection with AP2 I tried to use a newly
>> created virtual interface but the wpa_supplicant connection is not
>> successful. I'm not sure if this is a driver limitation or a driver
>> bug.
>
>
> I am not sure either. For the upstream brcmfmac we don't allow it at the
> moment. I am kinda surprised you were able to create the additional
> interface.

As you can see at [1], the .max_interface inside
sta_p2p_iface_combinations is set to 3 so I assume that's the reason
for being able to create a new virtual interface in the managed mode.
However, the driver  needs a new virtual interface only for the case
when during P2P-GO removal, P2P-GO is first changed to STA and later
only removed [2]. I don't think it's there for concurrent use with the
wlan0 interface or with the p2p interface. So, the concurrent usage of
wlan0 and a new vif seems to be a driver limitation, at least.

> There are a couple of things that I can think of here which may
> cause issues:
>
> - APs operating on different channels.
Actually I use the same AP both for the connection using the wla0
interface and the newly created virtual interface.
So, both interfaces are connecting using the same channel.

> - Both STA interfaces have powersave enabled.
How could I disable powesave mode?

> - Firmware/driver do not support multi STA.
>
Considering what I said above, it seems to be at least a driver
limitation.  How could I discover if it's also a firmware limitation?
(I CC-ed Ecco Park who has a lot of patches in the driver but he
didn't respond :)
>From my research, it seems that the firmware for BCM4330 (wireless
chip from Nexus 5) is closed-source. In this case, how does the driver
know how to communicate with the firmware? I assume there is an API
exposed by the firmware and the API is open-source. I would be really
grateful if you could give me more details about the communication
with the firmware.


> If APs are operating on the same channel it may work, but you probably won't
> have advantage of MP-TCP because of medium use.

Wow, I'm pleasantly surprised that you know about MP-TCP. That's
exactly what I will use. Connection to multiple APs was already
implemented on Linux and MP-TCP proved to have multiple advantages/
If we disregard WiFi interference between APs, the theoretically
optimal mobility solution is to always connect to every visible AP,
and let MPTCP handle load balancing at the transport layer: if an AP
has poor signal strength, its loss rate will be higher (because of
lower bandwidth and similar RTTs) and the MPTCP congestion controller
will simply migrate most of the traffic to the APs with better
connectivity to the client. This way, handover delays are eliminated
and the mobile enjoys continuous connectivity. But, yout're right,
medium use interferences can be a major issue, and you can read more
about at [3].

On Mon, Aug 24, 2015 at 2:08 PM, Gautam (Gautam Kumar) Shukla
<gautams@broadcom.com> wrote:
> I guess you can achieve the same by starting p2p client or gc and connect it
> with p2p go device. So you have 2 interface , one your wlan interface and
> another one is gc interface.
>

Yep, I already did this using a setup composed of two Nexus 5, a
laptop and an AP:
--------
wlan0: Nexus5_1 ----- AP ------ Laptop (on 5 Ghz)
First Nexus 5 is connected to the AP on 5Ghz. Also, the laptop is
connected with the AP on 5Ghz.

----
p2p: Nexus5_1 ---- (WiFi Direct on 2.4 Ghz) --- Nexus5_2 ---- AP (on 5
Ghz) --- Laptop:
First Nexus 5 is connected with the second Nexus 5 using WIFi Direct
on the 2.4 Ghz. Both the laptop and the second Nexus 5 are connected
to the AP on 5Ghz.
When I use just the wlan0 interface for an iperf test I get a high
throughput (~60Mbits/s). When I just connect the phones with WiFi
Direct (no iperf data transmitted on p2p) I get an iperf throughput
three times lower.
What are the possible explanations?

Do you have any numbers for the time needed for a switch between a the
2.4 Ghz and 5Ghz for bcmdhd?


Best regards,
Doru

[1] https://github.com/gdetal/mptcp_nexus5/blob/eb8eded8a6986e1ac94976ae206c74351abd0d1b/drivers/net/wireless/bcmdhd/wl_cfg80211.c#L195-L202
[2] https://github.com/gdetal/mptcp_nexus5/blob/eb8eded8a6986e1ac94976ae206c74351abd0d1b/drivers/net/wireless/bcmdhd/wl_cfg80211.c#L161-L163
[3] http://nets.cs.pub.ro/~costin/files/multi-wifi.pdf

Fwd: bcmdhd: Concurrent use of wlan0 and virtual interface in managed mode

[Gucea Doru]

Hello,

Thanks for helping me. I have some more questions related to parallel
usage of p2p and STA (station) interface.

I use a Nexus 5 device which has a BCM4339 [1] chip and it seems that
it's possible to transfer data in parallel using two connections: one
based on Wi-Fi direct(p2p, 2.4Ghz) and the other one based on a
connection to AP (wlan0, 5Ghz).

This is done by _time-sharing_ the wireless interface radio? From my
understanding the mac80211 layer have support for multiple virtual
interfaces and the switching between this interfaces is done inside
mac80211. Does the driver creates multiple virtual interfaces (one for
p2p and one for the connection to the AP) and the mac80211 manages the
time-sharing between them?

I am interested in understanding the code that decides how much the
radio should stay on a specific channel. I would be really grateful if
you could give me some details about the architecture.

[1] http://www.broadcom.com/press/release.php?id=s766879

Thanks,
Doru

Re: Fwd: bcmdhd: Concurrent use of wlan0 and virtual interface in managed mode

[Arend van Spriel]

On 09/07/2015 10:41 PM, Gucea Doru wrote:
> Hello,
>
> Thanks for helping me. I have some more questions related to parallel
> usage of p2p and STA (station) interface.
>
> I use a Nexus 5 device which has a BCM4339 [1] chip and it seems that
> it's possible to transfer data in parallel using two connections: one
> based on Wi-Fi direct(p2p, 2.4Ghz) and the other one based on a
> connection to AP (wlan0, 5Ghz).
>
> This is done by _time-sharing_ the wireless interface radio? From my
> understanding the mac80211 layer have support for multiple virtual
> interfaces and the switching between this interfaces is done inside
> mac80211. Does the driver creates multiple virtual interfaces (one for
> p2p and one for the connection to the AP) and the mac80211 manages the
> time-sharing between them?

Nope. The BCM4339 chip does not rely on mac80211. Instead, there is an 
802.11 stack running on the device, ie. in firmware. The driver creates 
the primary interface, ie. wlan0. Additional P2P interfaces are created 
by wpa_supplicant.

> I am interested in understanding the code that decides how much the
> radio should stay on a specific channel. I would be really grateful if
> you could give me some details about the architecture.

That code runs in firmware on the device and is proprietary. So I can 
not disclose any details.

Regards,
Arend

> [1] http://www.broadcom.com/press/release.php?id=s766879
>
> Thanks,
> Doru
> --
> To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Fwd: bcmdhd: Concurrent use of wlan0 and virtual interface in managed mode

[Gucea Doru]

On Tue, Sep 8, 2015 at 12:01 AM, Arend van Spriel <arend@broadcom.com> wrote:

> The BCM4339 chip does not rely on mac80211. Instead, there is an
> 802.11 stack running on the device, ie. in firmware. The driver creates the
> primary interface, ie. wlan0. Additional P2P interfaces are created by
> wpa_supplicant.
>

Hm, BCM4339 uses the bcmdhd driver.
So the bcmdhd driver is not a softmac[1] driver and the the 802.11
processing is not done in software, right?

[1] https://wireless.wiki.kernel.org/en/developers/documentation/glossary#softmac

Regards,
Doru

Re: Fwd: bcmdhd: Concurrent use of wlan0 and virtual interface in managed mode

[Arend van Spriel]

On 09/08/2015 09:50 AM, Gucea Doru wrote:
> On Tue, Sep 8, 2015 at 12:01 AM, Arend van Spriel <arend@broadcom.com> wrote:
>
>> The BCM4339 chip does not rely on mac80211. Instead, there is an
>> 802.11 stack running on the device, ie. in firmware. The driver creates the
>> primary interface, ie. wlan0. Additional P2P interfaces are created by
>> wpa_supplicant.
>>
>
> Hm, BCM4339 uses the bcmdhd driver.
> So the bcmdhd driver is not a softmac[1] driver and the the 802.11
> processing is not done in software, right?

Sure it is done in software. Just not on the host, ie. in linux kernel, 
but on the device.

Regards,
Arend

> [1] https://wireless.wiki.kernel.org/en/developers/documentation/glossary#softmac
>
> Regards,
> Doru
>