Re: Break-it testing for wifi

[Ben Greear <greearb@candelatech.com>]

On 11/28/2016 06:28 AM, Johannes Berg wrote:
> On Tue, 2016-11-22 at 08:59 -0800, Ben Greear wrote:
>>> Why would you do that? In order to test the AP implementation?
>>
>> Yes.  And really, you should be able to do similar things on the AP
>> to test stations, or on IBSS/Mesh devices, etc.  hostapd already has
>> some options to corrupt or drop a percentage of various management
>> frames.  Supplicant does not as far as I know.
>
> Yes, but that's a far different focus. I'm far more interested in
> testing *our* implementation(s), at which point I don't need to do it
> over the air for most cases, and can be much more efficient that way,
> etc.
>
> I also don't really see a point of doing anything over the air to test
> our implementations, apart from hitting firmware (but even then ...)

OTA is of primary importance to me, but I think any solution that works
for OTA would work for hwsim as well.

>
>>>> 2)  Randomly corrupt mgt frames in driver and/or mac80211 stack
>>>> and/or supplicant.
>>>
>>> I think fuzzing the input path for those frames would be more
>>> useful than just corrupting things.
>>
>> Random corruptions, especially by code that had at least some
>> understanding of management frames should be fast and easy to
>> use.  It would not be as good as a really clever fuzzer or hand-
>> crafted frames, but for many users, hand-crafting attacks would be
>> well beyond what they could ever accomplish.
>
> Define "user"?

System-test engineer in random company.  Like most of us, I am not working
for pure charity purposes :)

>
>> Currently, supplicant can (at least with some small patches that I
>> carry), add custom information elements to probe requests and
>> similar.  But, some things are built by mac80211 (rate-sets
>> advertised, for instance).  So, I was thinking of slightly extending
>> the API so that user-space could over-ride whatever mac80211 might
>> normally build itself.  That lets you more properly fuzz things from
>> user space.
>
> Why bother though, at that point? If you hook up some state transitions
> you can probably just send the frames from userspace. For these testing
> scenarios you can make assumptions about the hardware or query it
> beforehand, so there's no need to rely on mac80211 at all?

I don't want to re-implement supplicant, nor heavily modify it for this
effort.  I'm not sure exactly how important modifying IEs from user-space
would be for my effort, maybe existing API is enough with in-kernel fuzzer
I am thinking about writing.

>
>> Yes, but for ease of use, and to cover frames generated by mac80211,
>> I was thinking:
>>
>> echo 0.25 > /debug/.../wlan0/mgt_fuzzer
>>
>> The fuzzer would then corrupt 25% of the management frames.  And
>> instead of just randomly scribbling, it could also parse the frames
>> and do some more clever (and still pseudo-random) modifications to
>> the frames, like re-writing IEs with bad lengths, flipping bits in
>> specific portions of the frame we feel might find problems, etc.
>>
>> I think if the tool became useful, then it could grow more clever
>> over time.
>
> I'd be far more inclined to just add a tracepoint there at some spot
> and allow attaching BPF programs to it ... Or perhaps allow attaching
> BPF programs directly, if tracepoint BPF can't modify the data.
>
> Having any kind of logic here in the kernel space seems fairly useless
> since you'll want to change it often, etc.

Well, in-kernel seems best to me.  I will give it a try and see how
it works.

Thanks,
Ben

>
> johannes
>

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com