Any /n NICs that support APs and multiple STAs other than ath9k?

Any /n NICs that support APs and multiple STAs other than ath9k?

[Ben Greear]

I would like to have a different hardware/driver combination to try
to tie-break whether bugs are in mac80211 or in the ath9k driver/hardware.

I don't mind doing a bit of driver hacking so long as the basic support
is there (I think it's mostly the ability to set a BSSID mask and rxfilter
accordingly).

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Any /n NICs that support APs and multiple STAs other than ath9k?

[Luis R. Rodriguez]

On Wed, Oct 6, 2010 at 9:29 AM, Ben Greear <greearb@candelatech.com> wrote:
> I would like to have a different hardware/driver combination to try
> to tie-break whether bugs are in mac80211 or in the ath9k driver/hardware.
>
> I don't mind doing a bit of driver hacking so long as the basic support
> is there (I think it's mostly the ability to set a BSSID mask and rxfilter
> accordingly).

Your best bet is to test against mac80211_hwsim, that would rule out
any hardware. I think Jouni and Johannes have also a way to get the
devices to talk to each other. Perhaps some documentation of that on
the kernel Documentation/networking/mac80211_hwsim/

  Luis

Re: Any /n NICs that support APs and multiple STAs other than ath9k?

[Johannes Berg]

On Wed, 2010-10-06 at 11:45 -0700, Luis R. Rodriguez wrote:

> Your best bet is to test against mac80211_hwsim, that would rule out
> any hardware. I think Jouni and Johannes have also a way to get the
> devices to talk to each other. Perhaps some documentation of that on
> the kernel Documentation/networking/mac80211_hwsim/

You can either use network namespaces, or you can now somehow add local
routes (Patrick worked on that). But that's only needed if you need to
test IP connectivity at all between the interfaces, as long as you just
assoc etc. you don't need that.

johannes

Re: Any /n NICs that support APs and multiple STAs other than ath9k?

[Ben Greear]

On 10/06/2010 11:45 AM, Luis R. Rodriguez wrote:
> On Wed, Oct 6, 2010 at 9:29 AM, Ben Greear<greearb@candelatech.com>  wrote:
>> I would like to have a different hardware/driver combination to try
>> to tie-break whether bugs are in mac80211 or in the ath9k driver/hardware.
>>
>> I don't mind doing a bit of driver hacking so long as the basic support
>> is there (I think it's mostly the ability to set a BSSID mask and rxfilter
>> accordingly).
>
> Your best bet is to test against mac80211_hwsim, that would rule out
> any hardware. I think Jouni and Johannes have also a way to get the
> devices to talk to each other. Perhaps some documentation of that on
> the kernel Documentation/networking/mac80211_hwsim/

Ok.  I'm trying to hack slub to give me a better stack trace of where
the skb was deleted.  If that doesn't turn up anything obvious, then
I'll go read up on hwsim.

Are you aware of any DMA issues that might cause ath9k to write into
places it should not?  Previously, I've seen a lot of errors in
the logs about ath9k not being able to stop DMA in time, but I haven't
seen those while reproducing the memory corruption.


Thanks,
Ben

>
>    Luis


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Any /n NICs that support APs and multiple STAs other than ath9k?

[Ben Greear]

On 10/06/2010 11:48 AM, Johannes Berg wrote:
> On Wed, 2010-10-06 at 11:45 -0700, Luis R. Rodriguez wrote:
>
>> Your best bet is to test against mac80211_hwsim, that would rule out
>> any hardware. I think Jouni and Johannes have also a way to get the
>> devices to talk to each other. Perhaps some documentation of that on
>> the kernel Documentation/networking/mac80211_hwsim/
>
> You can either use network namespaces, or you can now somehow add local
> routes (Patrick worked on that). But that's only needed if you need to
> test IP connectivity at all between the interfaces, as long as you just
> assoc etc. you don't need that.

Oh, I'm using all of Patrick's stuff, and my application is a traffic
generator, so I have no trouble generating traffic to/from interfaces on
the same system.

That said, we're hitting these problems before we ever try to start
traffic most of the time, so it's not so important.

Thanks,
Ben

>
> johannes


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Any /n NICs that support APs and multiple STAs other than ath9k?

[Luis R. Rodriguez]

On Wed, Oct 6, 2010 at 11:49 AM, Ben Greear <greearb@candelatech.com> wrote:
> On 10/06/2010 11:45 AM, Luis R. Rodriguez wrote:
>>
>> On Wed, Oct 6, 2010 at 9:29 AM, Ben Greear<greearb@candelatech.com>
>>  wrote:
>>>
>>> I would like to have a different hardware/driver combination to try
>>> to tie-break whether bugs are in mac80211 or in the ath9k
>>> driver/hardware.
>>>
>>> I don't mind doing a bit of driver hacking so long as the basic support
>>> is there (I think it's mostly the ability to set a BSSID mask and
>>> rxfilter
>>> accordingly).
>>
>> Your best bet is to test against mac80211_hwsim, that would rule out
>> any hardware. I think Jouni and Johannes have also a way to get the
>> devices to talk to each other. Perhaps some documentation of that on
>> the kernel Documentation/networking/mac80211_hwsim/
>
> Ok.  I'm trying to hack slub to give me a better stack trace of where
> the skb was deleted.  If that doesn't turn up anything obvious, then
> I'll go read up on hwsim.
>
> Are you aware of any DMA issues that might cause ath9k to write into
> places it should not?  Previously, I've seen a lot of errors in
> the logs about ath9k not being able to stop DMA in time, but I haven't
> seen those while reproducing the memory corruption.

Yeah, well there have been a few bug reports but when we asked for
instructions how to reproduce we get nowhere. You are the first to
actually find some reproducible instructions. Forgive me but I haven't
had time yet to work on this though. Seems you have a good grasp of
things though and can easily reproduce so you likely are in a better
position right now to debug at the moment.

  Luis

Re: Any /n NICs that support APs and multiple STAs other than ath9k?

[Ben Greear]

On 10/06/2010 11:58 AM, Luis R. Rodriguez wrote:
> On Wed, Oct 6, 2010 at 11:49 AM, Ben Greear<greearb@candelatech.com>  wrote:
>> On 10/06/2010 11:45 AM, Luis R. Rodriguez wrote:
>>>
>>> On Wed, Oct 6, 2010 at 9:29 AM, Ben Greear<greearb@candelatech.com>
>>>   wrote:
>>>>
>>>> I would like to have a different hardware/driver combination to try
>>>> to tie-break whether bugs are in mac80211 or in the ath9k
>>>> driver/hardware.
>>>>
>>>> I don't mind doing a bit of driver hacking so long as the basic support
>>>> is there (I think it's mostly the ability to set a BSSID mask and
>>>> rxfilter
>>>> accordingly).
>>>
>>> Your best bet is to test against mac80211_hwsim, that would rule out
>>> any hardware. I think Jouni and Johannes have also a way to get the
>>> devices to talk to each other. Perhaps some documentation of that on
>>> the kernel Documentation/networking/mac80211_hwsim/
>>
>> Ok.  I'm trying to hack slub to give me a better stack trace of where
>> the skb was deleted.  If that doesn't turn up anything obvious, then
>> I'll go read up on hwsim.
>>
>> Are you aware of any DMA issues that might cause ath9k to write into
>> places it should not?  Previously, I've seen a lot of errors in
>> the logs about ath9k not being able to stop DMA in time, but I haven't
>> seen those while reproducing the memory corruption.
>
> Yeah, well there have been a few bug reports but when we asked for
> instructions how to reproduce we get nowhere. You are the first to
> actually find some reproducible instructions. Forgive me but I haven't
> had time yet to work on this though. Seems you have a good grasp of
> things though and can easily reproduce so you likely are in a better
> position right now to debug at the moment.

My understanding is that the bad DMA access could
corrupt things right out from under slub, and with no way for me
to possibly add any debugging to catch it in action.

I'm quite ignorant of hardware, DMA issues, etc, so if you have any
ideas on how to go about verifying bad DMA writes or not, I'm
interested.

Otherwise, I'll basically have to check and double check to make sure
we don't have a normal use-after-free with the skb, and if I can't find
anything, just assert that it's DMA issues and pretty much give up until
someone that understands that stuff has an idea to try.

Thanks,
Ben

>
>    Luis


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Any /n NICs that support APs and multiple STAs other than ath9k?

[Luis R. Rodriguez]

On Wed, Oct 6, 2010 at 12:08 PM, Ben Greear <greearb@candelatech.com> wrote:
> On 10/06/2010 11:58 AM, Luis R. Rodriguez wrote:
>>
>> On Wed, Oct 6, 2010 at 11:49 AM, Ben Greear<greearb@candelatech.com>
>>  wrote:
>>>
>>> On 10/06/2010 11:45 AM, Luis R. Rodriguez wrote:
>>>>
>>>> On Wed, Oct 6, 2010 at 9:29 AM, Ben Greear<greearb@candelatech.com>
>>>>  wrote:
>>>>>
>>>>> I would like to have a different hardware/driver combination to try
>>>>> to tie-break whether bugs are in mac80211 or in the ath9k
>>>>> driver/hardware.
>>>>>
>>>>> I don't mind doing a bit of driver hacking so long as the basic support
>>>>> is there (I think it's mostly the ability to set a BSSID mask and
>>>>> rxfilter
>>>>> accordingly).
>>>>
>>>> Your best bet is to test against mac80211_hwsim, that would rule out
>>>> any hardware. I think Jouni and Johannes have also a way to get the
>>>> devices to talk to each other. Perhaps some documentation of that on
>>>> the kernel Documentation/networking/mac80211_hwsim/
>>>
>>> Ok.  I'm trying to hack slub to give me a better stack trace of where
>>> the skb was deleted.  If that doesn't turn up anything obvious, then
>>> I'll go read up on hwsim.
>>>
>>> Are you aware of any DMA issues that might cause ath9k to write into
>>> places it should not?  Previously, I've seen a lot of errors in
>>> the logs about ath9k not being able to stop DMA in time, but I haven't
>>> seen those while reproducing the memory corruption.
>>
>> Yeah, well there have been a few bug reports but when we asked for
>> instructions how to reproduce we get nowhere. You are the first to
>> actually find some reproducible instructions. Forgive me but I haven't
>> had time yet to work on this though. Seems you have a good grasp of
>> things though and can easily reproduce so you likely are in a better
>> position right now to debug at the moment.
>
> My understanding is that the bad DMA access could
> corrupt things right out from under slub, and with no way for me
> to possibly add any debugging to catch it in action.
>
> I'm quite ignorant of hardware, DMA issues, etc, so if you have any
> ideas on how to go about verifying bad DMA writes or not, I'm
> interested.
>
> Otherwise, I'll basically have to check and double check to make sure
> we don't have a normal use-after-free with the skb, and if I can't find
> anything, just assert that it's DMA issues and pretty much give up until
> someone that understands that stuff has an idea to try.

You are able to reproduce easily so we should be able to now, we'll
get this bitch fixed.

  Luis