Tool to debug wifi pkt sniffs?

Tool to debug wifi pkt sniffs?

[Ben Greear]

Hello,

I often find myself wanting to figure out what equipment is to blame (and why)
in a wifi environment.

I am thinking writing a tool that would parse a pcap file and look at frames
in enough detail to flag block-ack bugs, rate-ctrl bugs, guess at the sniffer's
capture ability, etc.

Does anyone have anything already written that they would like to share, or know
of projects that might already do some of this?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Tool to debug wifi pkt sniffs?

[Toke Høiland-Jørgensen]

Ben Greear <greearb@candelatech.com> writes:

> Hello,
>
> I often find myself wanting to figure out what equipment is to blame (and why)
> in a wifi environment.
>
> I am thinking writing a tool that would parse a pcap file and look at frames
> in enough detail to flag block-ack bugs, rate-ctrl bugs, guess at the sniffer's
> capture ability, etc.
>
> Does anyone have anything already written that they would like to share, or know
> of projects that might already do some of this?

Not sure if this fits your criteria, but Sven's tool to create airtime
charts from packet sniffing data immediately came to mind:

https://github.com/cloudtrax/airtime-pie-chart

-Toke

Re: Tool to debug wifi pkt sniffs?

[Dave Taht]

On Wed, Oct 3, 2018 at 1:16 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>
> Ben Greear <greearb@candelatech.com> writes:
>
> > Hello,
> >
> > I often find myself wanting to figure out what equipment is to blame (and why)
> > in a wifi environment.
> >
> > I am thinking writing a tool that would parse a pcap file and look at frames
> > in enough detail to flag block-ack bugs, rate-ctrl bugs, guess at the sniffer's
> > capture ability, etc.
> >
> > Does anyone have anything already written that they would like to share, or know
> > of projects that might already do some of this?
>
> Not sure if this fits your criteria, but Sven's tool to create airtime
> charts from packet sniffing data immediately came to mind:
>
> https://github.com/cloudtrax/airtime-pie-chart

I have used that. Oy, it's a PITA. Some of kathie's code over here
(example: https://github.com/pollere/pping ) uses the slightly less
painful http://libtins.github.io/ library for parsing packets.


>
> -Toke



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: Tool to debug wifi pkt sniffs?

[Ben Greear]

On 10/03/2018 01:29 PM, Dave Taht wrote:
> On Wed, Oct 3, 2018 at 1:16 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>
>> Ben Greear <greearb@candelatech.com> writes:
>>
>>> Hello,
>>>
>>> I often find myself wanting to figure out what equipment is to blame (and why)
>>> in a wifi environment.
>>>
>>> I am thinking writing a tool that would parse a pcap file and look at frames
>>> in enough detail to flag block-ack bugs, rate-ctrl bugs, guess at the sniffer's
>>> capture ability, etc.
>>>
>>> Does anyone have anything already written that they would like to share, or know
>>> of projects that might already do some of this?
>>
>> Not sure if this fits your criteria, but Sven's tool to create airtime
>> charts from packet sniffing data immediately came to mind:
>>
>> https://github.com/cloudtrax/airtime-pie-chart
>
> I have used that. Oy, it's a PITA. Some of kathie's code over here
> (example: https://github.com/pollere/pping ) uses the slightly less
> painful http://libtins.github.io/ library for parsing packets.

I couldn't find anything that did what I wanted, so I wrote my own.

The (perl) code is in the wifi-diag directory of this public repo:

https://github.com/greearb/lanforge-scripts

The rest of the scripts in that repo are not related to the wifi-diag script, so just ignore those.

Here is example output for what I have so far:

https://www.candelatech.com/oss/wifi-diag/netgear-up-5s/index.html

The general idea is to get a performance test going, and then use tshark or similar
to grab a short sample (my script is slow, it can process only about 400 packets per second
on my desktop, so a 5 sec capture at full speed takes around 5 minutes to process),
and then pipe that decoded pcap into my script.

It tries to pay attention to latencies between block-ack and next AMPDU frame,
MCS distributions, packet-type distributions, retries, and other
such things.  I'm guessing tweaking wmm settings (or changing QoS in the generated traffic)
would be visible in this kind of metric, for instance.

The goal is to be able to answer the question of why one AP is faster or slower than another
when running the same test case.

Feedback (and even patches) is welcome...what other things can I report that would
be helpful?


Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Tool to debug wifi pkt sniffs?

[Dave Taht]

On Wed, Oct 10, 2018 at 10:10 AM Ben Greear <greearb@candelatech.com> wrote:
>
> On 10/03/2018 01:29 PM, Dave Taht wrote:
> > On Wed, Oct 3, 2018 at 1:16 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> >>
> >> Ben Greear <greearb@candelatech.com> writes:
> >>
> >>> Hello,
> >>>
> >>> I often find myself wanting to figure out what equipment is to blame (and why)
> >>> in a wifi environment.
> >>>
> >>> I am thinking writing a tool that would parse a pcap file and look at frames
> >>> in enough detail to flag block-ack bugs, rate-ctrl bugs, guess at the sniffer's
> >>> capture ability, etc.
> >>>
> >>> Does anyone have anything already written that they would like to share, or know
> >>> of projects that might already do some of this?
> >>
> >> Not sure if this fits your criteria, but Sven's tool to create airtime
> >> charts from packet sniffing data immediately came to mind:
> >>
> >> https://github.com/cloudtrax/airtime-pie-chart
> >
> > I have used that. Oy, it's a PITA. Some of kathie's code over here
> > (example: https://github.com/pollere/pping ) uses the slightly less
> > painful http://libtins.github.io/ library for parsing packets.
>
> I couldn't find anything that did what I wanted, so I wrote my own.
>
> The (perl) code is in the wifi-diag directory of this public repo:
>
> https://github.com/greearb/lanforge-scripts
>
> The rest of the scripts in that repo are not related to the wifi-diag script, so just ignore those.
>
> Here is example output for what I have so far:
>
> https://www.candelatech.com/oss/wifi-diag/netgear-up-5s/index.html

I *miss* writing in perl. :)

My guess from looking at that output that that was a udp flood test.
Do I win the internets?

>
> The general idea is to get a performance test going, and then use tshark or similar
> to grab a short sample (my script is slow, it can process only about 400 packets per second
> on my desktop, so a 5 sec capture at full speed takes around 5 minutes to process),
> and then pipe that decoded pcap into my script.
>
> It tries to pay attention to latencies between block-ack and next AMPDU frame,
> MCS distributions, packet-type distributions, retries, and other
> such things.  I'm guessing tweaking wmm settings (or changing QoS in the generated traffic)
> would be visible in this kind of metric, for instance.
>
> The goal is to be able to answer the question of why one AP is faster or slower than another
> when running the same test case.
>
> Feedback (and even patches) is welcome...what other things can I report that would
> be helpful?
>
>
> Thanks,
> Ben
>
> --
> Ben Greear <greearb@candelatech.com>
> Candela Technologies Inc  http://www.candelatech.com
>


-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: Tool to debug wifi pkt sniffs?

[Ben Greear]

On 10/10/2018 12:13 PM, Dave Taht wrote:
> On Wed, Oct 10, 2018 at 10:10 AM Ben Greear <greearb@candelatech.com> wrote:
>>
>> On 10/03/2018 01:29 PM, Dave Taht wrote:
>>> On Wed, Oct 3, 2018 at 1:16 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>>>
>>>> Ben Greear <greearb@candelatech.com> writes:
>>>>
>>>>> Hello,
>>>>>
>>>>> I often find myself wanting to figure out what equipment is to blame (and why)
>>>>> in a wifi environment.
>>>>>
>>>>> I am thinking writing a tool that would parse a pcap file and look at frames
>>>>> in enough detail to flag block-ack bugs, rate-ctrl bugs, guess at the sniffer's
>>>>> capture ability, etc.
>>>>>
>>>>> Does anyone have anything already written that they would like to share, or know
>>>>> of projects that might already do some of this?
>>>>
>>>> Not sure if this fits your criteria, but Sven's tool to create airtime
>>>> charts from packet sniffing data immediately came to mind:
>>>>
>>>> https://github.com/cloudtrax/airtime-pie-chart
>>>
>>> I have used that. Oy, it's a PITA. Some of kathie's code over here
>>> (example: https://github.com/pollere/pping ) uses the slightly less
>>> painful http://libtins.github.io/ library for parsing packets.
>>
>> I couldn't find anything that did what I wanted, so I wrote my own.
>>
>> The (perl) code is in the wifi-diag directory of this public repo:
>>
>> https://github.com/greearb/lanforge-scripts
>>
>> The rest of the scripts in that repo are not related to the wifi-diag script, so just ignore those.
>>
>> Here is example output for what I have so far:
>>
>> https://www.candelatech.com/oss/wifi-diag/netgear-up-5s/index.html
>
> I *miss* writing in perl. :)
>
> My guess from looking at that output that that was a udp flood test.
> Do I win the internets?

Yes, UDP upload test with 20 emulated stations, sending ~500 byte UDP frames.
One thing we notice in the case we are debugging, is
that the average time from transmitter station device receiving BA from the AP
to the transmitter station device putting the next AMPDU frame on air
is 0.728ms for the problem AP, and 0.448ms for the good AP.

I checked that the wmm config in the beacons for the two APs is the same.

I am at a loss as to what could cause this delay, other than possibly the problem
AP has a funky transmitter than puts a bit of extra noise on the air after it
is done transmitting a frame?

The problem AP also has 5% retransmits vs about 2% for the good AP, and problem AP
is typically using MCS8 instead of MCS9, but even so, I do not see how that would explain
the extra BA to AMPDU delay.

Thanks,
Ben


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Tool to debug wifi pkt sniffs?

[Dave Taht]

On Wed, Oct 10, 2018 at 1:44 PM Ben Greear <greearb@candelatech.com> wrote:
>
> On 10/10/2018 12:13 PM, Dave Taht wrote:
> > On Wed, Oct 10, 2018 at 10:10 AM Ben Greear <greearb@candelatech.com> wrote:
> >>
> >> On 10/03/2018 01:29 PM, Dave Taht wrote:
> >>> On Wed, Oct 3, 2018 at 1:16 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> >>>>
> >>>> Ben Greear <greearb@candelatech.com> writes:
> >>>>
> >>>>> Hello,
> >>>>>
> >>>>> I often find myself wanting to figure out what equipment is to blame (and why)
> >>>>> in a wifi environment.
> >>>>>
> >>>>> I am thinking writing a tool that would parse a pcap file and look at frames
> >>>>> in enough detail to flag block-ack bugs, rate-ctrl bugs, guess at the sniffer's
> >>>>> capture ability, etc.
> >>>>>
> >>>>> Does anyone have anything already written that they would like to share, or know
> >>>>> of projects that might already do some of this?
> >>>>
> >>>> Not sure if this fits your criteria, but Sven's tool to create airtime
> >>>> charts from packet sniffing data immediately came to mind:
> >>>>
> >>>> https://github.com/cloudtrax/airtime-pie-chart
> >>>
> >>> I have used that. Oy, it's a PITA. Some of kathie's code over here
> >>> (example: https://github.com/pollere/pping ) uses the slightly less
> >>> painful http://libtins.github.io/ library for parsing packets.
> >>
> >> I couldn't find anything that did what I wanted, so I wrote my own.
> >>
> >> The (perl) code is in the wifi-diag directory of this public repo:
> >>
> >> https://github.com/greearb/lanforge-scripts
> >>
> >> The rest of the scripts in that repo are not related to the wifi-diag script, so just ignore those.
> >>
> >> Here is example output for what I have so far:
> >>
> >> https://www.candelatech.com/oss/wifi-diag/netgear-up-5s/index.html
> >
> > I *miss* writing in perl. :)

I did take a quick look at the perl. It's been too long
> >
> > My guess from looking at that output that that was a udp flood test.
> > Do I win the internets?
>
> Yes, UDP upload test with 20 emulated stations, sending ~500 byte UDP frames.
> One thing we notice in the case we are debugging, is
> that the average time from transmitter station device receiving BA from the AP
> to the transmitter station device putting the next AMPDU frame on air
> is 0.728ms for the problem AP, and 0.448ms for the good AP.

I'm not big on averages. A cdf plot would show you if the delay was consistent
across the range or had a knee in it.

>
> I checked that the wmm config in the beacons for the two APs is the same.
>
> I am at a loss as to what could cause this delay, other than possibly the problem
> AP has a funky transmitter than puts a bit of extra noise on the air after it
> is done transmitting a frame?

A possible explanation would be garbage at one or more tried mcs rate
(not successfully captured).
minstrel at least tries multiple mcs rates.

>
> The problem AP also has 5% retransmits vs about 2% for the good AP, and problem AP
> is typically using MCS8 instead of MCS9, but even so, I do not see how that would explain
> the extra BA to AMPDU delay.

It's highly probable I'm misunderstanding you and would need to look
directly at the cap.

"typically using" says to me "more often trying the wrong rate"

>
> Thanks,
> Ben
>
>
> --
> Ben Greear <greearb@candelatech.com>
> Candela Technologies Inc  http://www.candelatech.com
>


-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740