[ath9k-devel] DMA TX Errors

[ath9k-devel] DMA TX Errors

[Paul Farrow]

Continuation of DMA errors saga...

Way back when, I bought a 9380 Sparklan card and it worked fine.  And 
it continues to work fine with my little duo board.  But I was getting 
really bad speeds from it (586kbps) so thought it was because I only had 
two of the antennas hooked up so decided to buy another 9280 card this 
time from Sparklan as I had had good experience with the 9380 one.   So 
last night I popped it into the motherboard and exactly the same problem 
that I had before with the apple 9280 card I have.

Ran it up as an access point and connect to it.  Normal browsing seems 
to be fine but as soon as I try to stream video (ie heavy usage) the DMA 
errors start and the machine is unusable.  I am using a 3.3.8-1 x86_64 
kernel on the Fedora FC16 distribution.

I guess the point of this post is to say I still have problems (with 
certain cards) and was wondering if there has been any developments with 
the DMA issues that people see and continue to see?

Irritated by it and keep thinking it will be fixed - I just cant seem 
to let it go plus the costs are creeping up :(

[ath9k-devel] DMA TX Errors

[Peter Stuge]

Paul Farrow wrote:
> Continuation of DMA errors saga...
..
> Irritated by it and keep thinking it will be fixed - I just cant
> seem to let it go plus the costs are creeping up :(

No I don't believe these errors will ever be fixed by Atheros.
Individual users seeing these errors is way too insignificant.

We are theoretically able to fix the problems ourselves, but the
economy of that is unjustifiable, and it is much better to simply
buy different hardware.

Unfortunately I can not recommend anything specific. :( I suggest a
buckshot approach: Buy a bag full of different cards and spend a day
or so evaluating each one in the context of your particular
mainboard. Then pick the winner. Keep in mind that you must
re-evaluate for any other mainboard.

If you like to support Atheros then as always the most modern
hardware you can possibly find is the best bet, because that will be
what the developers stopped working on most recently, so that code
will generally be better than the older code, and more likely to get
updated by future work. (Atheros works on drivers for hardware which
is being designed, not sold, and hardware lifetime is quite short.)


//Peter

[ath9k-devel] DMA TX Errors

[Adrian Chadd]

Hi,

It's still unclear what the problem is. Can you try booting into a
single-CPU mode and see if that helps?

Try disabling power saving operation (via an iw command) and see if that helps?

And Peter is somewhat right - it's unclear what is going on and we
don't see it here in any of the lab settings or engineers using ath9k.
Any time someone in the lab sees a problem we fix it. :-)

Which DMA errors in particular?


Adrian

[ath9k-devel] DMA TX Errors

[Paul Farrow]

Hi Adrian

The errors are the same that I experienced with the other 9280 card.  
Here is the bug that was reported 
https://bugzilla.kernel.org/show_bug.cgi?id=42673 some time ago.

Tried the power saving stuff before and it didn't work.  I have kinda 
lost the fire in my belly now and have the 9380 card working although 
the throughput seems slow.

If I ever upgrade my Jetway NF93R motherboard with a P8700 DUO on it 
then I might see a difference.  I definitely think its a problem with 
the type of hardware just as Peter said.  Have to buy a bunch of cards 
and just see if they work.

I was under the impression that Atheros were the better cards to get 
and the most supported for using with access points, does any one else 
have any other manufacturers to suggest that I might buy - that doesn't 
cost me an arm and a leg and is supported as an access point under the 
linux kernel.  Prefer 2 stream pcie cards.





On 15.06.2012 12:49, Adrian Chadd wrote:
> Hi,
>
> It's still unclear what the problem is. Can you try booting into a
> single-CPU mode and see if that helps?
>
> Try disabling power saving operation (via an iw command) and see if
> that helps?
>
> And Peter is somewhat right - it's unclear what is going on and we
> don't see it here in any of the lab settings or engineers using 
> ath9k.
> Any time someone in the lab sees a problem we fix it. :-)
>
> Which DMA errors in particular?
>
>
> Adrian
> _______________________________________________
> ath9k-devel mailing list
> ath9k-devel at lists.ath9k.org
> https://lists.ath9k.org/mailman/listinfo/ath9k-devel

[ath9k-devel] DMA TX Errors

[Paul Farrow]

I think you are completely correct from what I have experienced and 
viewed on this forum.  Its a shame really.  Do you have any experience 
of any other cards from any other vendors Peter?  I just want to have my 
own home made access point so that I can restrict what the kiddos are 
surfing.

Thanks

On 15.06.2012 09:21, Peter Stuge wrote:
> Paul Farrow wrote:
>> Continuation of DMA errors saga...
> ..
>> Irritated by it and keep thinking it will be fixed - I just cant
>> seem to let it go plus the costs are creeping up :(
>
> No I don't believe these errors will ever be fixed by Atheros.
> Individual users seeing these errors is way too insignificant.
>
> We are theoretically able to fix the problems ourselves, but the
> economy of that is unjustifiable, and it is much better to simply
> buy different hardware.
>
> Unfortunately I can not recommend anything specific. :( I suggest a
> buckshot approach: Buy a bag full of different cards and spend a day
> or so evaluating each one in the context of your particular
> mainboard. Then pick the winner. Keep in mind that you must
> re-evaluate for any other mainboard.
>
> If you like to support Atheros then as always the most modern
> hardware you can possibly find is the best bet, because that will be
> what the developers stopped working on most recently, so that code
> will generally be better than the older code, and more likely to get
> updated by future work. (Atheros works on drivers for hardware which
> is being designed, not sold, and hardware lifetime is quite short.)
>
>
> //Peter
> _______________________________________________
> ath9k-devel mailing list
> ath9k-devel at lists.ath9k.org
> https://lists.ath9k.org/mailman/listinfo/ath9k-devel

[ath9k-devel] DMA TX Errors

[Ben Greear]

On 06/22/2012 06:17 AM, Paul Farrow wrote:
> I think you are completely correct from what I have experienced and
> viewed on this forum.  Its a shame really.  Do you have any experience
> of any other cards from any other vendors Peter?  I just want to have my
> own home made access point so that I can restrict what the kiddos are
> surfing.

If you are running 3.4 or higher, I think my patches for showing IRQ
errors in debugfs are in place.  Can you post the contents of
the ath9k interrupt debugfs file after you see errors with your NIC?

cat /debug/ieee80211/wiphy0/ath9k/interrupt

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

[ath9k-devel] DMA TX Errors

[Paul Farrow]

Hi Ben

I am currently running 3.3.8 under Fedora 16.  It looks like that 
Kernel is not available in that distro yet 
http://fedoraproject.org/wiki/Kernel#Current_versions Do you have a 
kernel I could take and try and also could you give me brief 
instructions how to turn the debugging on.

Thanks again

Paul

On 22.06.2012 10:08, Ben Greear wrote:
> On 06/22/2012 06:17 AM, Paul Farrow wrote:
>> I think you are completely correct from what I have experienced and
>> viewed on this forum.  Its a shame really.  Do you have any 
>> experience
>> of any other cards from any other vendors Peter?  I just want to 
>> have my
>> own home made access point so that I can restrict what the kiddos 
>> are
>> surfing.
>
> If you are running 3.4 or higher, I think my patches for showing IRQ
> errors in debugfs are in place.  Can you post the contents of
> the ath9k interrupt debugfs file after you see errors with your NIC?
>
> cat /debug/ieee80211/wiphy0/ath9k/interrupt
>
> Thanks,
> Ben

[ath9k-devel] DMA TX Errors

[Ben Greear]

On 06/22/2012 07:17 AM, Paul Farrow wrote:
> Hi Ben
>
> I am currently running 3.3.8 under Fedora 16. It looks like that Kernel is not available in that distro yet
> http://fedoraproject.org/wiki/Kernel#Current_versions Do you have a kernel I could take and try and also could you give me brief instructions how to turn the
> debugging on.

Well, my 3.3.x kernel has all this debugging, and is stable in my testing
(well, I just upgraded to 3.3.8 yesterday and haven't tested yet, but
3.3.7 was rock solid).

git clone git://dmz2.candelatech.com/linux-3.3.dev.y

The upstream 3.5-rcX has the changes too, but it has been
quite flakey for me when testing wifi, so I don't suggest
using it yet.

Just enable debugfs for ath9k and mac80211 in kconfig,
that should be all that is needed.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

[ath9k-devel] DMA TX Errors

[Paul Farrow]

Hi Ben

Ok I will try to build your kernel over the weekend or early next week 
(have a busy schedule this weekend).

Will let you know.

Cheers

Paul


On 22.06.2012 11:13, Ben Greear wrote:
> On 06/22/2012 07:17 AM, Paul Farrow wrote:
>> Hi Ben
>>
>> I am currently running 3.3.8 under Fedora 16. It looks like that 
>> Kernel is not available in that distro yet
>> http://fedoraproject.org/wiki/Kernel#Current_versions Do you have a 
>> kernel I could take and try and also could you give me brief 
>> instructions how to turn the
>> debugging on.
>
> Well, my 3.3.x kernel has all this debugging, and is stable in my 
> testing
> (well, I just upgraded to 3.3.8 yesterday and haven't tested yet, but
> 3.3.7 was rock solid).
>
> git clone git://dmz2.candelatech.com/linux-3.3.dev.y
>
> The upstream 3.5-rcX has the changes too, but it has been
> quite flakey for me when testing wifi, so I don't suggest
> using it yet.
>
> Just enable debugfs for ath9k and mac80211 in kconfig,
> that should be all that is needed.
>
> Thanks,
> Ben

[ath9k-devel] DMA TX Errors

[Pannir]

Paul Farrow <ath9k <at> thefabfarrows.com> writes:

> 
> Continuation of DMA errors saga...
> 
> Way back when, I bought a 9380 Sparklan card and it worked fine.  And 
> it continues to work fine with my little duo board.  But I was getting 
> really bad speeds from it (586kbps) so thought it was because I only had 
> two of the antennas hooked up so decided to buy another 9280 card this 
> time from Sparklan as I had had good experience with the 9380 one.   So 
> last night I popped it into the motherboard and exactly the same problem 
> that I had before with the apple 9280 card I have.
> 
> Ran it up as an access point and connect to it.  Normal browsing seems 
> to be fine but as soon as I try to stream video (ie heavy usage) the DMA 
> errors start and the machine is unusable.  I am using a 3.3.8-1 x86_64 
> kernel on the Fedora FC16 distribution.
> 
> I guess the point of this post is to say I still have problems (with 
> certain cards) and was wondering if there has been any developments with 
> the DMA issues that people see and continue to see?
> 
> Irritated by it and keep thinking it will be fixed - I just cant seem 
> to let it go plus the costs are creeping up :(
> 


I have been seeing the same problem with the AR9280 on our MPC8315 based boards.
It does not happen on all our boards but the error seems to happen quicker on
some boards compared to others. I have not seen the issue on some boards. But
then again I have not tested any boards for more than 2 days. The problem
disappeared when I replaced the module with AR9382 from Unex (it is also 2x2).
The intermittent nature of the problem is making it difficult to troubleshoot on
the hardware level. Why is it happening on some boards and not others? Why is
AR9382 more forgiving than AR9280?

[ath9k-devel] DMA TX Errors

[Peter Stuge]

Pannir wrote:
> Why is it happening on some boards and not others? Why is AR9382
> more forgiving than AR9280?

Silicon bugs fixed or different code path in the driver with bugs
fixed or both.. Noone can answer, if they could I know that the
driver bugs would be fixed already.


//Peter

[ath9k-devel] DMA TX Errors

[Adrian Chadd]

On 23 June 2012 00:42, Peter Stuge <peter@stuge.se> wrote:
> Pannir wrote:
>> Why is it happening on some boards and not others? Why is AR9382
>> more forgiving than AR9280?
>
> Silicon bugs fixed or different code path in the driver with bugs
> fixed or both.. Noone can answer, if they could I know that the
> driver bugs would be fixed already.

They're different chips with different bits inside, they're on
different boards with different layouts, different components, etc,
etc.

I saw one AR9280 NIC which in my Lenovo T60 (don't ask) would
constantly throw all kinds of weird DMA errors, corrupted descriptors,
bus timeouts, etc.

I finally enabled SYNC interrupt counting and I found lots of PCIe bus
glue errors. I pointed this out to Ben Grear who wrote something
similar for ath9k and has found a few instances where the MAC seems to
be being fondled whilst it's supposed to be asleep.

I replaced with another AR5BXB92 (AR9280 dual-band) and everything
cleared up. All I can assume here is the NIC on that board was
marginal. The layout of the boards were slightly different, there
could've been some weird clock harmonics that _just_ annoyed things
the right way, or a busted cap somewhere on my misbehaving board.

I do suggest digging up that patch (I think he made reference to it in
the past) and running it, to at least eliminate PCIe bus errors as
part of the issue.

There are quite a few other reasons why the DMA sticking occurs - some
are hardware issues, some are software issues and race conditions. nbd
has been squishing them as fast as he can reproduce them.

Who knows what other weird and wonderful corner case PCIe bus issues
are creeping up? The only way to investigate those is with a PCIe bus
analyser.

HTH,


Adrian

[ath9k-devel] DMA TX Errors

[Pannir]

Adrian Chadd <adrian <at> freebsd.org> writes:

> I do suggest digging up that patch (I think he made reference to it in
> the past) and running it, to at least eliminate PCIe bus errors as
> part of the issue.
> 
> There are quite a few other reasons why the DMA sticking occurs - some
> are hardware issues, some are software issues and race conditions. nbd
> has been squishing them as fast as he can reproduce them.
> 
> Who knows what other weird and wonderful corner case PCIe bus issues
> are creeping up? The only way to investigate those is with a PCIe bus
> analyser.
> 
> HTH,
> 
> Adrian
> 

Is this the patch you are referring to?
http://comments.gmane.org/gmane.linux.drivers.ath9k.devel/8494

Any suggestion for a good bus analyser?