iwlwifi micorcode errors and hardware resets problems

iwlwifi micorcode errors and hardware resets problems

[Stanislaw Gruszka]

We have those bug reports on RH bugzilla, and various other iwlwifi
problems, but those have some common scenario:

https://bugzilla.redhat.com/show_bug.cgi?id=979873
https://bugzilla.redhat.com/show_bug.cgi?id=990536
https://bugzilla.redhat.com/show_bug.cgi?id=994322
https://bugzilla.redhat.com/show_bug.cgi?id=996502
https://bugzilla.redhat.com/show_bug.cgi?id=969610
https://bugzilla.redhat.com/show_bug.cgi?id=999053

First there is Microcode error or other firmware problem, what make we
do iwl_trans_stop_device() which set trans->state to IWL_TRANS_NO_FW.
If mac80211 has some pending works/timers i.e. ieee80211_sta_work(),
it calls drv and we trigger on of those warnings

        WARN_ONCE(trans->state != IWL_TRANS_FW_ALIVE,
                  "%s bad state = %d", __func__, trans->state);

on some trans operation.

I think this can be fixed by first do quiesce on mac80211 (for example
by ieee80211_sta_quiesce() procedure, which I removed but can be added
back) and then do iwlagn_prepare_restart() & ieee80211_restart_hw().

Though I'm not sure if it's worth to do this since those warnings are
indicator for malfunctioning iwlwifi firmware (or driver not correctly
communicating with firmware), which will be unnoticed if not reported
automatically by tool like ABRT. But perhaps we could add WARN_ONCE on
Microcode error condition ?

Additionally I wonder if there is good way to debug those microcode
problems (I mean root of the problem), since most of them happen at
random after hours or days uptime?

Thoughts?

Stanislaw

Re: iwlwifi micorcode errors and hardware resets problems

[Luca Coelho]

On Fri, 2013-08-23 at 11:08 +0200, Stanislaw Gruszka wrote:
> We have those bug reports on RH bugzilla, and various other iwlwifi
> problems, but those have some common scenario:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=979873
> https://bugzilla.redhat.com/show_bug.cgi?id=990536
> https://bugzilla.redhat.com/show_bug.cgi?id=994322
> https://bugzilla.redhat.com/show_bug.cgi?id=996502
> https://bugzilla.redhat.com/show_bug.cgi?id=969610
> https://bugzilla.redhat.com/show_bug.cgi?id=999053
> 
> First there is Microcode error or other firmware problem, what make we
> do iwl_trans_stop_device() which set trans->state to IWL_TRANS_NO_FW.
> If mac80211 has some pending works/timers i.e. ieee80211_sta_work(),
> it calls drv and we trigger on of those warnings
> 
>         WARN_ONCE(trans->state != IWL_TRANS_FW_ALIVE,
>                   "%s bad state = %d", __func__, trans->state);
> 
> on some trans operation.
> 
> I think this can be fixed by first do quiesce on mac80211 (for example
> by ieee80211_sta_quiesce() procedure, which I removed but can be added
> back) and then do iwlagn_prepare_restart() & ieee80211_restart_hw().
> 
> Though I'm not sure if it's worth to do this since those warnings are
> indicator for malfunctioning iwlwifi firmware (or driver not correctly
> communicating with firmware), which will be unnoticed if not reported
> automatically by tool like ABRT. But perhaps we could add WARN_ONCE on
> Microcode error condition ?

I think that, if the quiesce really helps with this, we should do it.
This WARN is a side-effect of the problem and not the problem itself.
If you want to catch the FW problem, there should be a WARN elsewhere.

I'm not saying that this WARN should be removed and I also think it's a
good idea to change it to WARN_ON_ONCE().  I just think that, if there's
anything the driver can do to keep things running nicely, we should do
it.

In any case, I'll leave this for Johannes, Emmanuel or some of the other
guys with more experience in iwlwifi to comment. ;)

--
Cheers,
Luca.

RE: [Ilw] Re: iwlwifi micorcode errors and hardware resets problems

[Grumbach, Emmanuel]

PiBPbiBGcmksIDIwMTMtMDgtMjMgYXQgMTE6MDggKzAyMDAsIFN0YW5pc2xhdyBHcnVzemthIHdy
b3RlOg0KPiA+IFdlIGhhdmUgdGhvc2UgYnVnIHJlcG9ydHMgb24gUkggYnVnemlsbGEsIGFuZCB2
YXJpb3VzIG90aGVyIGl3bHdpZmkNCj4gPiBwcm9ibGVtcywgYnV0IHRob3NlIGhhdmUgc29tZSBj
b21tb24gc2NlbmFyaW86DQo+ID4NCj4gPiBodHRwczovL2J1Z3ppbGxhLnJlZGhhdC5jb20vc2hv
d19idWcuY2dpP2lkPTk3OTg3Mw0KPiA+IGh0dHBzOi8vYnVnemlsbGEucmVkaGF0LmNvbS9zaG93
X2J1Zy5jZ2k/aWQ9OTkwNTM2DQo+ID4gaHR0cHM6Ly9idWd6aWxsYS5yZWRoYXQuY29tL3Nob3df
YnVnLmNnaT9pZD05OTQzMjINCj4gPiBodHRwczovL2J1Z3ppbGxhLnJlZGhhdC5jb20vc2hvd19i
dWcuY2dpP2lkPTk5NjUwMg0KPiA+IGh0dHBzOi8vYnVnemlsbGEucmVkaGF0LmNvbS9zaG93X2J1
Zy5jZ2k/aWQ9OTY5NjEwDQo+ID4gaHR0cHM6Ly9idWd6aWxsYS5yZWRoYXQuY29tL3Nob3dfYnVn
LmNnaT9pZD05OTkwNTMNCj4gPg0KPiA+IEZpcnN0IHRoZXJlIGlzIE1pY3JvY29kZSBlcnJvciBv
ciBvdGhlciBmaXJtd2FyZSBwcm9ibGVtLCB3aGF0IG1ha2Ugd2UNCj4gPiBkbyBpd2xfdHJhbnNf
c3RvcF9kZXZpY2UoKSB3aGljaCBzZXQgdHJhbnMtPnN0YXRlIHRvIElXTF9UUkFOU19OT19GVy4N
Cj4gPiBJZiBtYWM4MDIxMSBoYXMgc29tZSBwZW5kaW5nIHdvcmtzL3RpbWVycyBpLmUuIGllZWU4
MDIxMV9zdGFfd29yaygpLA0KPiA+IGl0IGNhbGxzIGRydiBhbmQgd2UgdHJpZ2dlciBvbiBvZiB0
aG9zZSB3YXJuaW5ncw0KPiA+DQo+ID4gICAgICAgICBXQVJOX09OQ0UodHJhbnMtPnN0YXRlICE9
IElXTF9UUkFOU19GV19BTElWRSwNCj4gPiAgICAgICAgICAgICAgICAgICAiJXMgYmFkIHN0YXRl
ID0gJWQiLCBfX2Z1bmNfXywgdHJhbnMtPnN0YXRlKTsNCj4gPg0KPiA+IG9uIHNvbWUgdHJhbnMg
b3BlcmF0aW9uLg0KPiA+DQo+ID4gSSB0aGluayB0aGlzIGNhbiBiZSBmaXhlZCBieSBmaXJzdCBk
byBxdWllc2NlIG9uIG1hYzgwMjExIChmb3IgZXhhbXBsZQ0KPiA+IGJ5IGllZWU4MDIxMV9zdGFf
cXVpZXNjZSgpIHByb2NlZHVyZSwgd2hpY2ggSSByZW1vdmVkIGJ1dCBjYW4gYmUgYWRkZWQNCj4g
PiBiYWNrKSBhbmQgdGhlbiBkbyBpd2xhZ25fcHJlcGFyZV9yZXN0YXJ0KCkgJiBpZWVlODAyMTFf
cmVzdGFydF9odygpLg0KPiA+DQo+ID4gVGhvdWdoIEknbSBub3Qgc3VyZSBpZiBpdCdzIHdvcnRo
IHRvIGRvIHRoaXMgc2luY2UgdGhvc2Ugd2FybmluZ3MgYXJlDQo+ID4gaW5kaWNhdG9yIGZvciBt
YWxmdW5jdGlvbmluZyBpd2x3aWZpIGZpcm13YXJlIChvciBkcml2ZXIgbm90IGNvcnJlY3RseQ0K
PiA+IGNvbW11bmljYXRpbmcgd2l0aCBmaXJtd2FyZSksIHdoaWNoIHdpbGwgYmUgdW5ub3RpY2Vk
IGlmIG5vdCByZXBvcnRlZA0KPiA+IGF1dG9tYXRpY2FsbHkgYnkgdG9vbCBsaWtlIEFCUlQuIEJ1
dCBwZXJoYXBzIHdlIGNvdWxkIGFkZCBXQVJOX09OQ0Ugb24NCj4gPiBNaWNyb2NvZGUgZXJyb3Ig
Y29uZGl0aW9uID8NCj4gDQo+IEkgdGhpbmsgdGhhdCwgaWYgdGhlIHF1aWVzY2UgcmVhbGx5IGhl
bHBzIHdpdGggdGhpcywgd2Ugc2hvdWxkIGRvIGl0Lg0KPiBUaGlzIFdBUk4gaXMgYSBzaWRlLWVm
ZmVjdCBvZiB0aGUgcHJvYmxlbSBhbmQgbm90IHRoZSBwcm9ibGVtIGl0c2VsZi4NCj4gSWYgeW91
IHdhbnQgdG8gY2F0Y2ggdGhlIEZXIHByb2JsZW0sIHRoZXJlIHNob3VsZCBiZSBhIFdBUk4gZWxz
ZXdoZXJlLg0KPiANCj4gSSdtIG5vdCBzYXlpbmcgdGhhdCB0aGlzIFdBUk4gc2hvdWxkIGJlIHJl
bW92ZWQgYW5kIEkgYWxzbyB0aGluayBpdCdzIGEgZ29vZA0KPiBpZGVhIHRvIGNoYW5nZSBpdCB0
byBXQVJOX09OX09OQ0UoKS4gIEkganVzdCB0aGluayB0aGF0LCBpZiB0aGVyZSdzIGFueXRoaW5n
DQo+IHRoZSBkcml2ZXIgY2FuIGRvIHRvIGtlZXAgdGhpbmdzIHJ1bm5pbmcgbmljZWx5LCB3ZSBz
aG91bGQgZG8gaXQuDQo+IA0KPiBJbiBhbnkgY2FzZSwgSSdsbCBsZWF2ZSB0aGlzIGZvciBKb2hh
bm5lcywgRW1tYW51ZWwgb3Igc29tZSBvZiB0aGUgb3RoZXIgZ3V5cw0KPiB3aXRoIG1vcmUgZXhw
ZXJpZW5jZSBpbiBpd2x3aWZpIHRvIGNvbW1lbnQuIDspDQoNCllvdSBnb3QgbXkgQUNLIGhlcmUu
IFdlIGhhdmUgc29tZSBpc3N1ZXMgd2l0aCBmaXJtd2FyZS4gT2J2aW91c2x5LiBCdXQgbW9zdCBv
ZiB0aGUgaXNzdWVzIGFyZSB2ZXJ5IGhhcmQgdG8gZGVidWcuIE1vc3Qgb2YgdGhlbSBJIGFtIHNl
ZWluZyBhcmUgbW9yZSBsaWtlIHRoZSBmaXJtd2FyZSBnZXR0aW5nIHN0dWNrIGV0Yy4uLiAgQWZ0
ZXIgYWxsIHRoaXMgaXMgd2h5IHdlIGhhdmUgYSByZXN0YXJ0IGZsb3csIGZvciB0aGVzZSBpc3N1
ZXMgdGhhdCBkb24ndCBoYXBwZW4gZnJlcXVlbnRseSBidXQgYXJlIHNvbHZhYmxlIGJ5IGEgcmVz
ZXQuIFNpbmNlIHdlIHN0aWxsIHdhbnQgdG8ga25vdyB3aGVuIHRoaXMga2luZCBvZiB0aGluZ3Mg
aGFwcGVuLCBhZGRpbmcgYSBXQVJOX09OX09OQ0Ugc291bmRzIGdvb2QuDQo=