A good way to get CSI info to user-space?

A good way to get CSI info to user-space?

[Ben Greear]

Hello,

I have been poking at ath10k to get some CSI reporting working.
The CSI reports are a bit over 2k bytes in length, and they could
happen quite often.  As far as I can tell, there is no good way in
the kernel to make this available.

I was thinking maybe a callback into the mac80211 stack, which would
then send a netlink event might be a useful way to implement this?

Any other suggestions on this?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: A good way to get CSI info to user-space?

[Johannes Berg]

Hi,

> I have been poking at ath10k to get some CSI reporting working.
> The CSI reports are a bit over 2k bytes in length, and they could
> happen quite often.  As far as I can tell, there is no good way in
> the kernel to make this available.
> 
> I was thinking maybe a callback into the mac80211 stack, which would
> then send a netlink event might be a useful way to implement this?

The CSI data has variable length [1] but it's fundamentally always tied
to a specific frame and as such we've always attached it to that frame
using a radiotap vendor namespace.

You can easily implement that in a mac80211 driver since it has support
for that via RX_FLAG_RADIOTAP_VENDOR_DATA and the associated struct
ieee80211_vendor_radiotap that you put into the skb's head.

Why should anything else be needed?

johannes

[1] mostly depending on the bandwidth

Re: A good way to get CSI info to user-space?

[Ben Greear]

On 10/23/2017 01:30 PM, Johannes Berg wrote:
> Hi,
>
>> I have been poking at ath10k to get some CSI reporting working.
>> The CSI reports are a bit over 2k bytes in length, and they could
>> happen quite often.  As far as I can tell, there is no good way in
>> the kernel to make this available.
>>
>> I was thinking maybe a callback into the mac80211 stack, which would
>> then send a netlink event might be a useful way to implement this?
>
> The CSI data has variable length [1] but it's fundamentally always tied
> to a specific frame and as such we've always attached it to that frame
> using a radiotap vendor namespace.
>
> You can easily implement that in a mac80211 driver since it has support
> for that via RX_FLAG_RADIOTAP_VENDOR_DATA and the associated struct
> ieee80211_vendor_radiotap that you put into the skb's head.
>
> Why should anything else be needed?

So this would only show up in user-space in something like a pkt-capture?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: A good way to get CSI info to user-space?

[Johannes Berg]

On Mon, 2017-10-23 at 13:59 -0700, Ben Greear wrote:

> > The CSI data has variable length [1] but it's fundamentally always tied
> > to a specific frame and as such we've always attached it to that frame
> > using a radiotap vendor namespace.
> > 
> > You can easily implement that in a mac80211 driver since it has support
> > for that via RX_FLAG_RADIOTAP_VENDOR_DATA and the associated struct
> > ieee80211_vendor_radiotap that you put into the skb's head.
> > 
> > Why should anything else be needed?
> 
> So this would only show up in user-space in something like a pkt-capture?

Sure. But since you can easily add virtual monitor interfaces at any
time, that doesn't really mean anything.

johannes

Re: A good way to get CSI info to user-space?

[Ben Greear]

On 10/23/2017 10:54 PM, Johannes Berg wrote:
> On Mon, 2017-10-23 at 13:59 -0700, Ben Greear wrote:
>
>>> The CSI data has variable length [1] but it's fundamentally always tied
>>> to a specific frame and as such we've always attached it to that frame
>>> using a radiotap vendor namespace.
>>>
>>> You can easily implement that in a mac80211 driver since it has support
>>> for that via RX_FLAG_RADIOTAP_VENDOR_DATA and the associated struct
>>> ieee80211_vendor_radiotap that you put into the skb's head.
>>>
>>> Why should anything else be needed?
>>
>> So this would only show up in user-space in something like a pkt-capture?
>
> Sure. But since you can easily add virtual monitor interfaces at any
> time, that doesn't really mean anything.

Adding a monitor device is a pretty big hammer on ath10k, at least, and
probably not something one would want running continuously in a production
environment.

The CSI data is one part, but tx-beamforming is another area of interest.

If I get a chance, I'll try to add a way to pass some of this opaque
info up through netlink events.  I'm thinking something like:

mac80211_nl_send_opaque_event(mac80211-handle, int data-type, u8* data, int data_len);

data-type would be:
NLD_ATH_CSI_DATA = 0,
NLD_ATH_TXBF_DATA = 1,
NLD_PRIVATE_START = 5000,
....


'iw events' could be made to dump raw hex for external programs to decode, or maybe
decode it a bit itself in some cases.  And, third-party programs can open a netlink
monitor socket and get the data directly.

Thanks,
Ben


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: A good way to get CSI info to user-space?

[Johannes Berg]

On Tue, 2017-10-24 at 06:36 -0700, Ben Greear wrote:
> 
> On 10/23/2017 10:54 PM, Johannes Berg wrote:
> > On Mon, 2017-10-23 at 13:59 -0700, Ben Greear wrote:
> > 
> > > > The CSI data has variable length [1] but it's fundamentally always tied
> > > > to a specific frame and as such we've always attached it to that frame
> > > > using a radiotap vendor namespace.
> > > > 
> > > > You can easily implement that in a mac80211 driver since it has support
> > > > for that via RX_FLAG_RADIOTAP_VENDOR_DATA and the associated struct
> > > > ieee80211_vendor_radiotap that you put into the skb's head.
> > > > 
> > > > Why should anything else be needed?
> > > 
> > > So this would only show up in user-space in something like a pkt-capture?
> > 
> > Sure. But since you can easily add virtual monitor interfaces at any
> > time, that doesn't really mean anything.
> 
> Adding a monitor device is a pretty big hammer on ath10k, at least, and
> probably not something one would want running continuously in a production
> environment.

We keep having this discussion. You need to fix this, it should be
really simple to fix this - just remove all checks for CONF_MONITOR
from ath10k and make it use WANT_MONITOR_VIF instead if it doesn't
already.

> The CSI data is one part, but tx-beamforming is another area of interest.
> 
> If I get a chance, I'll try to add a way to pass some of this opaque
> info up through netlink events.  I'm thinking something like:
> 
> mac80211_nl_send_opaque_event(mac80211-handle, int data-type, u8* data, int data_len);

I will not accept this upstream.

johannes

Re: A good way to get CSI info to user-space?

[Ben Greear]

On 10/24/2017 06:38 AM, Johannes Berg wrote:
> On Tue, 2017-10-24 at 06:36 -0700, Ben Greear wrote:
>>
>> On 10/23/2017 10:54 PM, Johannes Berg wrote:
>>> On Mon, 2017-10-23 at 13:59 -0700, Ben Greear wrote:
>>>
>>>>> The CSI data has variable length [1] but it's fundamentally always tied
>>>>> to a specific frame and as such we've always attached it to that frame
>>>>> using a radiotap vendor namespace.
>>>>>
>>>>> You can easily implement that in a mac80211 driver since it has support
>>>>> for that via RX_FLAG_RADIOTAP_VENDOR_DATA and the associated struct
>>>>> ieee80211_vendor_radiotap that you put into the skb's head.
>>>>>
>>>>> Why should anything else be needed?
>>>>
>>>> So this would only show up in user-space in something like a pkt-capture?
>>>
>>> Sure. But since you can easily add virtual monitor interfaces at any
>>> time, that doesn't really mean anything.
>>
>> Adding a monitor device is a pretty big hammer on ath10k, at least, and
>> probably not something one would want running continuously in a production
>> environment.
>
> We keep having this discussion. You need to fix this, it should be
> really simple to fix this - just remove all checks for CONF_MONITOR
> from ath10k and make it use WANT_MONITOR_VIF instead if it doesn't
> already.

I don't recall having this discussion, but even if you can do a
non-promiscuous monitor mode, you would have to end up having a bpf
socket filter and packet socket, right?

Wouldn't that have a noticeable effect on performance on modest sized
AP hardware?

>> The CSI data is one part, but tx-beamforming is another area of interest.
>>
>> If I get a chance, I'll try to add a way to pass some of this opaque
>> info up through netlink events.  I'm thinking something like:
>>
>> mac80211_nl_send_opaque_event(mac80211-handle, int data-type, u8* data, int data_len);
>
> I will not accept this upstream.

Thanks for letting me know up front, at least.

Anyone know if there is a useful way to stream events from debugfs and/or sysfs w/out having to busy-poll
on it?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: A good way to get CSI info to user-space?

[Johannes Berg]

On Tue, 2017-10-24 at 06:50 -0700, Ben Greear wrote:

> > We keep having this discussion. You need to fix this, it should be
> > really simple to fix this - just remove all checks for CONF_MONITOR
> > from ath10k and make it use WANT_MONITOR_VIF instead if it doesn't
> > already.
> 
> I don't recall having this discussion, 

I'm pretty sure I had this discussion about ath10k not liking virtual
monitor, but maybe it was with somebody else.

> but even if you can do a
> non-promiscuous monitor mode, you would have to end up having a bpf
> socket filter and packet socket, right?

True.

> Wouldn't that have a noticeable effect on performance on modest sized
> AP hardware?

Yes. I was planning to add the wifi BPF to fix that:
https://git.kernel.org/pub/scm/linux/kernel/git/jberg/mac80211-next.git/log/?h=bpf

You shouldn't think that building netlink messages is free though.
There's a lot more locking etc. going on there.

> > > mac80211_nl_send_opaque_event(mac80211-handle, int data-type, u8* data, int data_len);
> > 
> > I will not accept this upstream.
> 
> Thanks for letting me know up front, at least.
> 
> Anyone know if there is a useful way to stream events from debugfs
> and/or sysfs w/out having to busy-poll on it?

There's relayfs, which is kinda built for that. You could even use
netlink vendor events, but I don't really think netlink is appropriate
here.

johannes

Re: A good way to get CSI info to user-space?

[Ben Greear]

On 10/24/2017 06:55 AM, Johannes Berg wrote:
> On Tue, 2017-10-24 at 06:50 -0700, Ben Greear wrote:
>
>>> We keep having this discussion. You need to fix this, it should be
>>> really simple to fix this - just remove all checks for CONF_MONITOR
>>> from ath10k and make it use WANT_MONITOR_VIF instead if it doesn't
>>> already.
>>
>> I don't recall having this discussion,
>
> I'm pretty sure I had this discussion about ath10k not liking virtual
> monitor, but maybe it was with somebody else.
>
>> but even if you can do a
>> non-promiscuous monitor mode, you would have to end up having a bpf
>> socket filter and packet socket, right?
>
> True.
>
>> Wouldn't that have a noticeable effect on performance on modest sized
>> AP hardware?
>
> Yes. I was planning to add the wifi BPF to fix that:
> https://git.kernel.org/pub/scm/linux/kernel/git/jberg/mac80211-next.git/log/?h=bpf
>
> You shouldn't think that building netlink messages is free though.
> There's a lot more locking etc. going on there.

CSI info is quite rare (probe-responses only at this point),
so even if each report is somewhat costly, that
will still be a lot less over-all work than running any kind of filter on
every frame.

I'm not sure how often tx-beamforming data is made available, but probably
it is somewhat rare compared to normal data packets, at least in ath10k.

>>>> mac80211_nl_send_opaque_event(mac80211-handle, int data-type, u8* data, int data_len);
>>>
>>> I will not accept this upstream.
>>
>> Thanks for letting me know up front, at least.
>>
>> Anyone know if there is a useful way to stream events from debugfs
>> and/or sysfs w/out having to busy-poll on it?
>
> There's relayfs, which is kinda built for that. You could even use
> netlink vendor events, but I don't really think netlink is appropriate
> here.

I'll look into the netlink vendor events.

Thanks,
Ben

>
> johannes
>


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: A good way to get CSI info to user-space?

[Zefir Kurtisi]

Hi Ben,

On 10/24/2017 03:55 PM, Johannes Berg wrote:
> On Tue, 2017-10-24 at 06:50 -0700, Ben Greear wrote:
> [...]
>> Anyone know if there is a useful way to stream events from debugfs
>> and/or sysfs w/out having to busy-poll on it?
> 
> There's relayfs, which is kinda built for that. You could even use
> netlink vendor events, but I don't really think netlink is appropriate
> here.
> 
This is what I would also suggest, alas it is not clear if the events need to be
processed realtime.

The relayfs is used to stream the spectral data from ath9k (and afaik also from
ath10k) to userspace. Very easy to implement: kernel pushes TLVs into the relayfs,
userspace pulls them.

It is 'realtime-enough' to display realtime spectral graphs, but since you are
reading the stream in buffer-slices, you have to wait for a slice to fill up
before it is available at userspace (or you flush every TLV at kernel side, which
kills efficiency).

That is: if you need to immediately process events, relayfs is maybe not the right
choice, if you can process them in bulk, that's the most efficient way to go.

Cheers,
Zefir

> johannes
> 

Re: A good way to get CSI info to user-space?

[Ben Greear]

On 10/25/2017 07:30 AM, Zefir Kurtisi wrote:
> Hi Ben,
>
> On 10/24/2017 03:55 PM, Johannes Berg wrote:
>> On Tue, 2017-10-24 at 06:50 -0700, Ben Greear wrote:
>> [...]
>>> Anyone know if there is a useful way to stream events from debugfs
>>> and/or sysfs w/out having to busy-poll on it?
>>
>> There's relayfs, which is kinda built for that. You could even use
>> netlink vendor events, but I don't really think netlink is appropriate
>> here.
>>
> This is what I would also suggest, alas it is not clear if the events need to be
> processed realtime.
>
> The relayfs is used to stream the spectral data from ath9k (and afaik also from
> ath10k) to userspace. Very easy to implement: kernel pushes TLVs into the relayfs,
> userspace pulls them.
>
> It is 'realtime-enough' to display realtime spectral graphs, but since you are
> reading the stream in buffer-slices, you have to wait for a slice to fill up
> before it is available at userspace (or you flush every TLV at kernel side, which
> kills efficiency).
>
> That is: if you need to immediately process events, relayfs is maybe not the right
> choice, if you can process them in bulk, that's the most efficient way to go.

The netlink vendor API seems to do exactly what I was hoping for earlier,
so I have started poking at that.

There is at least one other set of 'per-peer tx-beamforming' data that might be worth
exporting via the same mechanism.

Any particular reason why relayfs would be better than netlink?

Thanks,
Ben


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: A good way to get CSI info to user-space?

[Zefir Kurtisi]

On 10/25/2017 04:43 PM, Ben Greear wrote:
> 
> 
> On 10/25/2017 07:30 AM, Zefir Kurtisi wrote:
>> Hi Ben,
>>
>> On 10/24/2017 03:55 PM, Johannes Berg wrote:
>>> On Tue, 2017-10-24 at 06:50 -0700, Ben Greear wrote:
>>> [...]
>>>> Anyone know if there is a useful way to stream events from debugfs
>>>> and/or sysfs w/out having to busy-poll on it?
>>>
>>> There's relayfs, which is kinda built for that. You could even use
>>> netlink vendor events, but I don't really think netlink is appropriate
>>> here.
>>>
>> This is what I would also suggest, alas it is not clear if the events need to be
>> processed realtime.
>>
>> The relayfs is used to stream the spectral data from ath9k (and afaik also from
>> ath10k) to userspace. Very easy to implement: kernel pushes TLVs into the relayfs,
>> userspace pulls them.
>>
>> It is 'realtime-enough' to display realtime spectral graphs, but since you are
>> reading the stream in buffer-slices, you have to wait for a slice to fill up
>> before it is available at userspace (or you flush every TLV at kernel side, which
>> kills efficiency).
>>
>> That is: if you need to immediately process events, relayfs is maybe not the right
>> choice, if you can process them in bulk, that's the most efficient way to go.
> 
> The netlink vendor API seems to do exactly what I was hoping for earlier,
> so I have started poking at that.
> 
> There is at least one other set of 'per-peer tx-beamforming' data that might be worth
> exporting via the same mechanism.
> 
> Any particular reason why relayfs would be better than netlink?
> 
> 

'Better' depends on type of the data. relayfs is better for continuous and
high-bandwidth data. With spectral there are up to 80k measurements per second
with each TLV holding ~70-150 bytes (depending on HT20/40) - that won't be doable
as netlink events.

If the netlink vendor API suffices for the CSI data, no need to change.

Aside from data type dependency, relayfs has at least some noteworthy positive
aspects:
* works lock-free at kernel side
* independent of any API at userspace
  => just open the file from {C, python, shell, ...} and process the data


Cheers,
Zefir