Re: iwl4965: "MAC is in deep sleep!" freezes

Re: iwl4965: "MAC is in deep sleep!" freezes

[Jonathan Nieder]

Hi,

Christoph Anton Mitterer wrote[1]:

> When I bring the WLAN interface up, after some time (usually just
> minutes) I get the following error (during which the system
> completely freezes for some 10-20s)
[...]
> kernel: [  951.087153] iwl4965 0000:14:00.0: Error sending REPLY_SCAN_CMD: time out after 500ms.
> wpa_supplicant[13032]: Failed to initiate AP scan.
> kernel: [  952.089208] ------------[ cut here ]------------
> kernel: [  952.090089] WARNING: at drivers/net/wireless/iwlegacy/iwl-tx.c:483 iwl_legacy_enqueue_hcmd+0x3c7/0x3f0()

The warning is from v3.0-rc2~7^2~16^2~216 (iwlegacy: fix enqueue hcmd
race conditions, 2011-04-28):

| -	spin_lock_irqsave(&priv->hcmd_lock, flags);
| -
| -	/* If this is a huge cmd, mark the huge flag also on the meta.flags
| -	 * of the _original_ cmd. This is used for DMA mapping clean up.
| -	 */
| -	if (cmd->flags & CMD_SIZE_HUGE) {
| -		idx = iwl_legacy_get_cmd_index(q, q->write_ptr, 0);
| -		txq->meta[idx].flags = CMD_SIZE_HUGE;
| -	}
| -
| 	idx = iwl_legacy_get_cmd_index(q, q->write_ptr, cmd->flags & CMD_SIZE_HUGE);
| 	out_cmd = txq->cmd[idx];
| 	out_meta = &txq->meta[idx];
|  
| +	if (WARN_ON(out_meta->flags & CMD_MAPPED)) {
| +		spin_unlock_irqrestore(&priv->hcmd_lock, flags);
| +		return -ENOSPC;
| +	}
| +

More details are at [1].  Stanislav, thoughts?  Is this a bug, and if
so, any ideas for tracking it down?

Thanks,
Jonathan

[1] http://bugs.debian.org/636355

Re: iwl4965: "MAC is in deep sleep!" freezes

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 353 bytes --]

Hi guys :)

If there's any quick fix (like reverting the patch above) I'd be happy
to test it,... cause I'm on a conference next week,.. and having no Wifi
there is annyoing.

btw: I might be, but I'm not sure about this anymore, that the issue has
already appeared with 2.6.39... but again... not sure on this, forgot
it.


Cheers,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5677 bytes --]

Re: iwl4965: "MAC is in deep sleep!" freezes

[Stanislaw Gruszka]

Hello.

On Sat, Sep 03, 2011 at 02:45:22PM +0200, Christoph Anton Mitterer wrote:
> If there's any quick fix (like reverting the patch above) I'd be happy
> to test it,
No patch is available for that problem as far as I know. I requested for
verbose debug messages for now.

> ... cause I'm on a conference next week,.. and having no Wifi
> there is annyoing.
Perhaps you can downgrade kernel to 2.6.38 to make wifi works.

Stanislaw

Re: iwl4965: "MAC is in deep sleep!" freezes

[Stanislaw Gruszka]

Hello

On Fri, Sep 02, 2011 at 06:38:06PM -0500, Jonathan Nieder wrote:
> Christoph Anton Mitterer wrote[1]:
> 
> > When I bring the WLAN interface up, after some time (usually just
> > minutes) I get the following error (during which the system
> > completely freezes for some 10-20s)
> [...]
> > kernel: [  951.087153] iwl4965 0000:14:00.0: Error sending REPLY_SCAN_CMD: time out after 500ms.
> > wpa_supplicant[13032]: Failed to initiate AP scan.
> > kernel: [  952.089208] ------------[ cut here ]------------
> > kernel: [  952.090089] WARNING: at drivers/net/wireless/iwlegacy/iwl-tx.c:483 iwl_legacy_enqueue_hcmd+0x3c7/0x3f0()
> 
> The warning is from v3.0-rc2~7^2~16^2~216 (iwlegacy: fix enqueue hcmd
> race conditions, 2011-04-28):
> 
> | -	spin_lock_irqsave(&priv->hcmd_lock, flags);
> | -
> | -	/* If this is a huge cmd, mark the huge flag also on the meta.flags
> | -	 * of the _original_ cmd. This is used for DMA mapping clean up.
> | -	 */
> | -	if (cmd->flags & CMD_SIZE_HUGE) {
> | -		idx = iwl_legacy_get_cmd_index(q, q->write_ptr, 0);
> | -		txq->meta[idx].flags = CMD_SIZE_HUGE;
> | -	}
> | -
> | 	idx = iwl_legacy_get_cmd_index(q, q->write_ptr, cmd->flags & CMD_SIZE_HUGE);
> | 	out_cmd = txq->cmd[idx];
> | 	out_meta = &txq->meta[idx];
> |  
> | +	if (WARN_ON(out_meta->flags & CMD_MAPPED)) {
> | +		spin_unlock_irqrestore(&priv->hcmd_lock, flags);
> | +		return -ENOSPC;
> | +	}
> | +
> 
> More details are at [1].  Stanislav, thoughts?  Is this a bug, and if
> so, any ideas for tracking it down?
It's a bug. But this looks like some different problem than is showed in
[1]

> Thanks,
> Jonathan
> 
> [1] http://bugs.debian.org/636355

Based on:
Jul 25 01:14:59 heisenberg kernel: [  121.551098] iwl4965 0000:14:00.0:
Queue 4 stuck for 2000 ms.
Jul 25 01:14:59 heisenberg kernel: [  121.555226] iwl4965 0000:14:00.0:
On demand firmware reload
Jul 25 01:15:12 heisenberg kernel: [  121.560305] iwl4965 0000:14:00.0:
MAC is in deep sleep!.  CSR_GP_CNTRL = 0x080033D8

this look like firmware hang. This happen just after module load, that's
good news, because it allow to log relative small amount debug messages
to see what possibly driver do wrong to crash firmware.

Please configure syslog to log kernel debug messages. Not sure how to do
this in debian, see  "Configure syslog to log kernel debug messages" in
https://fedoraproject.org/wiki/DebugWireless for corresponding fedora
config. Then reload driver:

modprobe -r iwl4965
echo > /var/log/kernel
modprobe iwl4965 debug=0x47ffffff

and send me /var/log/kernel including messages from iwl4965 module
load to first "MAC is in deep sleep" message.

Note this require kernel compiled with 
CONFIG_IWLWIFI_LEGACY_DEBUG=y

Thanks
Stanislaw

Re: iwl4965: "MAC is in deep sleep!" freezes

[Christoph Anton Mitterer]

On Mon, 5 Sep 2011 11:19:18 +0200, Stanislaw Gruszka <sgruszka@redhat.com>
wrote:
> this look like firmware hang. This happen just after module load, that's
> good news, because it allow to log relative small amount debug messages
> to see what possibly driver do wrong to crash firmware.
Note, that the effects of - usually but not always - frozen system while
the errors are printed (sometimes keyboard input seems still be possible
during that) ... happen over and over again... say about every  10 minutes
or so.
Not just the first time the modules is loaded.
Is the firmware loaded over and over again?


> Please configure syslog to log kernel debug messages.
Will mail it later.


Cheers,
Chris.

Re: iwl4965: "MAC is in deep sleep!" freezes

[Stanislaw Gruszka]

On Mon, Sep 05, 2011 at 09:42:01AM +0000, Christoph Anton Mitterer wrote:
> On Mon, 5 Sep 2011 11:19:18 +0200, Stanislaw Gruszka <sgruszka@redhat.com>
> wrote:
> > this look like firmware hang. This happen just after module load, that's
> > good news, because it allow to log relative small amount debug messages
> > to see what possibly driver do wrong to crash firmware.
> Note, that the effects of - usually but not always - frozen system while
> the errors are printed (sometimes keyboard input seems still be possible
> during that) ... happen over and over again... say about every  10 minutes
> or so.
> Not just the first time the modules is loaded.
> Is the firmware loaded over and over again?
I think is not, but even if is, kernel should not crash. Can you grab
crash logs using photo camera, serial cable or netconsole, to allow to 
see where exactly crash happen and fix it.

> > Please configure syslog to log kernel debug messages.
> Will mail it later.

Firmware hang seems to happen when REPLY_LEDS_CMD command is send.
No idea if attached patch could help or not, but is worth to try it.
It prevent to send LED command asynchronously, when possibly some
other commands are pending in firmware.

If it does not help, please provide the same way captured debug
messages from 2.6.38 kernel, where drivers works. I'll compare
it with broken driver logs to figure out what broken driver
do differently.

Thanks
Stanislaw

Re: iwl4965: "MAC is in deep sleep!" freezes

[Stanislaw Gruszka]

[-- Attachment #1: Type: text/plain, Size: 1594 bytes --]

On Thu, Sep 08, 2011 at 10:49:13AM +0200, Stanislaw Gruszka wrote:
> On Mon, Sep 05, 2011 at 09:42:01AM +0000, Christoph Anton Mitterer wrote:
> > On Mon, 5 Sep 2011 11:19:18 +0200, Stanislaw Gruszka <sgruszka@redhat.com>
> > wrote:
> > > this look like firmware hang. This happen just after module load, that's
> > > good news, because it allow to log relative small amount debug messages
> > > to see what possibly driver do wrong to crash firmware.
> > Note, that the effects of - usually but not always - frozen system while
> > the errors are printed (sometimes keyboard input seems still be possible
> > during that) ... happen over and over again... say about every  10 minutes
> > or so.
> > Not just the first time the modules is loaded.
> > Is the firmware loaded over and over again?
> I think is not, but even if is, kernel should not crash. Can you grab
> crash logs using photo camera, serial cable or netconsole, to allow to 
> see where exactly crash happen and fix it.
> 
> > > Please configure syslog to log kernel debug messages.
> > Will mail it later.
> 
> Firmware hang seems to happen when REPLY_LEDS_CMD command is send.
> No idea if attached patch could help or not, but is worth to try it.
> It prevent to send LED command asynchronously, when possibly some
> other commands are pending in firmware.
> 
> If it does not help, please provide the same way captured debug
> messages from 2.6.38 kernel, where drivers works. I'll compare
> it with broken driver logs to figure out what broken driver
> do differently.

Sorry, forgot attch the patch, do it now.

Stanislaw

[-- Attachment #2: led_work.patch --]
[-- Type: text/plain, Size: 4319 bytes --]

diff --git a/drivers/net/wireless/iwlegacy/iwl-3945-led.c b/drivers/net/wireless/iwlegacy/iwl-3945-led.c
index abd9235..127ffd6 100644
--- a/drivers/net/wireless/iwlegacy/iwl-3945-led.c
+++ b/drivers/net/wireless/iwlegacy/iwl-3945-led.c
@@ -52,8 +52,7 @@ static int iwl3945_send_led_cmd(struct iwl_priv *priv,
 		.id = REPLY_LEDS_CMD,
 		.len = sizeof(struct iwl_led_cmd),
 		.data = led_cmd,
-		.flags = CMD_ASYNC,
-		.callback = NULL,
+		.flags = CMD_SYNC,
 	};
 
 	return iwl_legacy_send_cmd(priv, &cmd);
diff --git a/drivers/net/wireless/iwlegacy/iwl-4965-led.c b/drivers/net/wireless/iwlegacy/iwl-4965-led.c
index 26d324e..2545c19 100644
--- a/drivers/net/wireless/iwlegacy/iwl-4965-led.c
+++ b/drivers/net/wireless/iwlegacy/iwl-4965-led.c
@@ -51,8 +51,7 @@ iwl4965_send_led_cmd(struct iwl_priv *priv, struct iwl_led_cmd *led_cmd)
 		.id = REPLY_LEDS_CMD,
 		.len = sizeof(struct iwl_led_cmd),
 		.data = led_cmd,
-		.flags = CMD_ASYNC,
-		.callback = NULL,
+		.flags = CMD_SYNC,
 	};
 	u32 reg;
 
diff --git a/drivers/net/wireless/iwlegacy/iwl-dev.h b/drivers/net/wireless/iwlegacy/iwl-dev.h
index 9c786ed..2623c79 100644
--- a/drivers/net/wireless/iwlegacy/iwl-dev.h
+++ b/drivers/net/wireless/iwlegacy/iwl-dev.h
@@ -1211,6 +1211,8 @@ struct iwl_priv {
 	struct delayed_work alive_start;
 	struct delayed_work scan_check;
 
+	struct work_struct led_work;
+
 	/* TX Power */
 	s8 tx_power_user_lmt;
 	s8 tx_power_device_lmt;
diff --git a/drivers/net/wireless/iwlegacy/iwl-led.c b/drivers/net/wireless/iwlegacy/iwl-led.c
index bda0d61..2152d87 100644
--- a/drivers/net/wireless/iwlegacy/iwl-led.c
+++ b/drivers/net/wireless/iwlegacy/iwl-led.c
@@ -106,13 +106,6 @@ static int iwl_legacy_led_cmd(struct iwl_priv *priv,
 		.id = IWL_LED_LINK,
 		.interval = IWL_DEF_LED_INTRVL
 	};
-	int ret;
-
-	if (!test_bit(STATUS_READY, &priv->status))
-		return -EBUSY;
-
-	if (priv->blink_on == on && priv->blink_off == off)
-		return 0;
 
 	if (off == 0) {
 		/* led is SOLID_ON */
@@ -126,24 +119,25 @@ static int iwl_legacy_led_cmd(struct iwl_priv *priv,
 	led_cmd.off = iwl_legacy_blink_compensation(priv, off,
 				priv->cfg->base_params->led_compensation);
 
-	ret = priv->cfg->ops->led->cmd(priv, &led_cmd);
-	if (!ret) {
-		priv->blink_on = on;
-		priv->blink_off = off;
-	}
-	return ret;
+	return priv->cfg->ops->led->cmd(priv, &led_cmd);
 }
 
 static void iwl_legacy_led_brightness_set(struct led_classdev *led_cdev,
 				   enum led_brightness brightness)
 {
 	struct iwl_priv *priv = container_of(led_cdev, struct iwl_priv, led);
-	unsigned long on = 0;
+	unsigned long on = (brightness > 0) ? IWL_LED_SOLID : 0;
 
-	if (brightness > 0)
-		on = IWL_LED_SOLID;
+	if (!test_bit(STATUS_READY, &priv->status))
+		return;
+
+	if (priv->blink_on == on && priv->blink_off == 0)
+		return;
+
+	priv->blink_on = on;
+	priv->blink_off = 0;
 
-	iwl_legacy_led_cmd(priv, on, 0);
+	queue_work(priv->workqueue, &priv->led_work);
 }
 
 static int iwl_legacy_led_blink_set(struct led_classdev *led_cdev,
@@ -152,7 +146,28 @@ static int iwl_legacy_led_blink_set(struct led_classdev *led_cdev,
 {
 	struct iwl_priv *priv = container_of(led_cdev, struct iwl_priv, led);
 
-	return iwl_legacy_led_cmd(priv, *delay_on, *delay_off);
+	if (!test_bit(STATUS_READY, &priv->status))
+		return -EBUSY;
+
+	if (priv->blink_on == *delay_on && priv->blink_off == *delay_off)
+		return 0;
+
+	/* FIXME: Need spinlock to sync with possibly currently running led work? */
+	priv->blink_on = *delay_on;
+	priv->blink_off = *delay_off;
+
+	queue_work(priv->workqueue, &priv->led_work);
+	
+	return 0;
+}
+
+static void iwl_legacy_bg_led(struct work_struct *work)
+{
+	struct iwl_priv *priv = container_of(work, struct iwl_priv, led_work);
+
+	mutex_lock(&priv->mutex);
+	iwl_legacy_led_cmd(priv, priv->blink_on, priv->blink_off);
+	mutex_unlock(&priv->mutex);
 }
 
 void iwl_legacy_leds_init(struct iwl_priv *priv)
@@ -160,6 +175,8 @@ void iwl_legacy_leds_init(struct iwl_priv *priv)
 	int mode = led_mode;
 	int ret;
 
+	INIT_WORK(&priv->led_work, iwl_legacy_bg_led);
+
 	if (mode == IWL_LED_DEFAULT)
 		mode = priv->cfg->led_mode;
 
@@ -202,5 +219,7 @@ void iwl_legacy_leds_exit(struct iwl_priv *priv)
 
 	led_classdev_unregister(&priv->led);
 	kfree(priv->led.name);
+
+	cancel_work_sync(&priv->led_work);
 }
 EXPORT_SYMBOL(iwl_legacy_leds_exit);

Re: iwl4965: "MAC is in deep sleep!" freezes

[Stanislaw Gruszka]

On Thu, Sep 08, 2011 at 06:19:56PM +0200, Stanislaw Gruszka wrote:
> On Thu, Sep 08, 2011 at 10:49:13AM +0200, Stanislaw Gruszka wrote:
> > On Mon, Sep 05, 2011 at 09:42:01AM +0000, Christoph Anton Mitterer wrote:
> > > On Mon, 5 Sep 2011 11:19:18 +0200, Stanislaw Gruszka <sgruszka@redhat.com>
> > > wrote:
> > > > this look like firmware hang. This happen just after module load, that's
> > > > good news, because it allow to log relative small amount debug messages
> > > > to see what possibly driver do wrong to crash firmware.
> > > Note, that the effects of - usually but not always - frozen system while
> > > the errors are printed (sometimes keyboard input seems still be possible
> > > during that) ... happen over and over again... say about every  10 minutes
> > > or so.
> > > Not just the first time the modules is loaded.
> > > Is the firmware loaded over and over again?
> > I think is not, but even if is, kernel should not crash. Can you grab
> > crash logs using photo camera, serial cable or netconsole, to allow to 
> > see where exactly crash happen and fix it.
> > 
> > > > Please configure syslog to log kernel debug messages.
> > > Will mail it later.
> > 
> > Firmware hang seems to happen when REPLY_LEDS_CMD command is send.
> > No idea if attached patch could help or not, but is worth to try it.
> > It prevent to send LED command asynchronously, when possibly some
> > other commands are pending in firmware.
> > 
> > If it does not help, please provide the same way captured debug
> > messages from 2.6.38 kernel, where drivers works. I'll compare
> > it with broken driver logs to figure out what broken driver
> > do differently.
> 
> Sorry, forgot attch the patch, do it now.

Any feedback here?

Stanislaw

> diff --git a/drivers/net/wireless/iwlegacy/iwl-3945-led.c b/drivers/net/wireless/iwlegacy/iwl-3945-led.c
> index abd9235..127ffd6 100644
> --- a/drivers/net/wireless/iwlegacy/iwl-3945-led.c
> +++ b/drivers/net/wireless/iwlegacy/iwl-3945-led.c
> @@ -52,8 +52,7 @@ static int iwl3945_send_led_cmd(struct iwl_priv *priv,
>  		.id = REPLY_LEDS_CMD,
>  		.len = sizeof(struct iwl_led_cmd),
>  		.data = led_cmd,
> -		.flags = CMD_ASYNC,
> -		.callback = NULL,
> +		.flags = CMD_SYNC,
>  	};
>  
>  	return iwl_legacy_send_cmd(priv, &cmd);
> diff --git a/drivers/net/wireless/iwlegacy/iwl-4965-led.c b/drivers/net/wireless/iwlegacy/iwl-4965-led.c
> index 26d324e..2545c19 100644
> --- a/drivers/net/wireless/iwlegacy/iwl-4965-led.c
> +++ b/drivers/net/wireless/iwlegacy/iwl-4965-led.c
> @@ -51,8 +51,7 @@ iwl4965_send_led_cmd(struct iwl_priv *priv, struct iwl_led_cmd *led_cmd)
>  		.id = REPLY_LEDS_CMD,
>  		.len = sizeof(struct iwl_led_cmd),
>  		.data = led_cmd,
> -		.flags = CMD_ASYNC,
> -		.callback = NULL,
> +		.flags = CMD_SYNC,
>  	};
>  	u32 reg;
>  
> diff --git a/drivers/net/wireless/iwlegacy/iwl-dev.h b/drivers/net/wireless/iwlegacy/iwl-dev.h
> index 9c786ed..2623c79 100644
> --- a/drivers/net/wireless/iwlegacy/iwl-dev.h
> +++ b/drivers/net/wireless/iwlegacy/iwl-dev.h
> @@ -1211,6 +1211,8 @@ struct iwl_priv {
>  	struct delayed_work alive_start;
>  	struct delayed_work scan_check;
>  
> +	struct work_struct led_work;
> +
>  	/* TX Power */
>  	s8 tx_power_user_lmt;
>  	s8 tx_power_device_lmt;
> diff --git a/drivers/net/wireless/iwlegacy/iwl-led.c b/drivers/net/wireless/iwlegacy/iwl-led.c
> index bda0d61..2152d87 100644
> --- a/drivers/net/wireless/iwlegacy/iwl-led.c
> +++ b/drivers/net/wireless/iwlegacy/iwl-led.c
> @@ -106,13 +106,6 @@ static int iwl_legacy_led_cmd(struct iwl_priv *priv,
>  		.id = IWL_LED_LINK,
>  		.interval = IWL_DEF_LED_INTRVL
>  	};
> -	int ret;
> -
> -	if (!test_bit(STATUS_READY, &priv->status))
> -		return -EBUSY;
> -
> -	if (priv->blink_on == on && priv->blink_off == off)
> -		return 0;
>  
>  	if (off == 0) {
>  		/* led is SOLID_ON */
> @@ -126,24 +119,25 @@ static int iwl_legacy_led_cmd(struct iwl_priv *priv,
>  	led_cmd.off = iwl_legacy_blink_compensation(priv, off,
>  				priv->cfg->base_params->led_compensation);
>  
> -	ret = priv->cfg->ops->led->cmd(priv, &led_cmd);
> -	if (!ret) {
> -		priv->blink_on = on;
> -		priv->blink_off = off;
> -	}
> -	return ret;
> +	return priv->cfg->ops->led->cmd(priv, &led_cmd);
>  }
>  
>  static void iwl_legacy_led_brightness_set(struct led_classdev *led_cdev,
>  				   enum led_brightness brightness)
>  {
>  	struct iwl_priv *priv = container_of(led_cdev, struct iwl_priv, led);
> -	unsigned long on = 0;
> +	unsigned long on = (brightness > 0) ? IWL_LED_SOLID : 0;
>  
> -	if (brightness > 0)
> -		on = IWL_LED_SOLID;
> +	if (!test_bit(STATUS_READY, &priv->status))
> +		return;
> +
> +	if (priv->blink_on == on && priv->blink_off == 0)
> +		return;
> +
> +	priv->blink_on = on;
> +	priv->blink_off = 0;
>  
> -	iwl_legacy_led_cmd(priv, on, 0);
> +	queue_work(priv->workqueue, &priv->led_work);
>  }
>  
>  static int iwl_legacy_led_blink_set(struct led_classdev *led_cdev,
> @@ -152,7 +146,28 @@ static int iwl_legacy_led_blink_set(struct led_classdev *led_cdev,
>  {
>  	struct iwl_priv *priv = container_of(led_cdev, struct iwl_priv, led);
>  
> -	return iwl_legacy_led_cmd(priv, *delay_on, *delay_off);
> +	if (!test_bit(STATUS_READY, &priv->status))
> +		return -EBUSY;
> +
> +	if (priv->blink_on == *delay_on && priv->blink_off == *delay_off)
> +		return 0;
> +
> +	/* FIXME: Need spinlock to sync with possibly currently running led work? */
> +	priv->blink_on = *delay_on;
> +	priv->blink_off = *delay_off;
> +
> +	queue_work(priv->workqueue, &priv->led_work);
> +	
> +	return 0;
> +}
> +
> +static void iwl_legacy_bg_led(struct work_struct *work)
> +{
> +	struct iwl_priv *priv = container_of(work, struct iwl_priv, led_work);
> +
> +	mutex_lock(&priv->mutex);
> +	iwl_legacy_led_cmd(priv, priv->blink_on, priv->blink_off);
> +	mutex_unlock(&priv->mutex);
>  }
>  
>  void iwl_legacy_leds_init(struct iwl_priv *priv)
> @@ -160,6 +175,8 @@ void iwl_legacy_leds_init(struct iwl_priv *priv)
>  	int mode = led_mode;
>  	int ret;
>  
> +	INIT_WORK(&priv->led_work, iwl_legacy_bg_led);
> +
>  	if (mode == IWL_LED_DEFAULT)
>  		mode = priv->cfg->led_mode;
>  
> @@ -202,5 +219,7 @@ void iwl_legacy_leds_exit(struct iwl_priv *priv)
>  
>  	led_classdev_unregister(&priv->led);
>  	kfree(priv->led.name);
> +
> +	cancel_work_sync(&priv->led_work);
>  }
>  EXPORT_SYMBOL(iwl_legacy_leds_exit);

Re: iwl4965: "MAC is in deep sleep!" freezes

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 1996 bytes --]

Hi.

Sorry for completely forgetting about this... (for so long)

I've tried your patch now against 3.1, and it seems to not help.

Not sure whether trying with 2.6.38 helps.
It "stopped" working within the very same kernel.

The old mainboard got broken and was replaced by a new one (of the same
type),... but perhaps the wifi controller is of a new revision or so?!


Cheers,
Chris.


On Thu, 2011-09-08 at 10:49 +0200, Stanislaw Gruszka wrote:
> On Mon, Sep 05, 2011 at 09:42:01AM +0000, Christoph Anton Mitterer wrote:
> > On Mon, 5 Sep 2011 11:19:18 +0200, Stanislaw Gruszka <sgruszka@redhat.com>
> > wrote:
> > > this look like firmware hang. This happen just after module load, that's
> > > good news, because it allow to log relative small amount debug messages
> > > to see what possibly driver do wrong to crash firmware.
> > Note, that the effects of - usually but not always - frozen system while
> > the errors are printed (sometimes keyboard input seems still be possible
> > during that) ... happen over and over again... say about every  10 minutes
> > or so.
> > Not just the first time the modules is loaded.
> > Is the firmware loaded over and over again?
> I think is not, but even if is, kernel should not crash. Can you grab
> crash logs using photo camera, serial cable or netconsole, to allow to 
> see where exactly crash happen and fix it.
> 
> > > Please configure syslog to log kernel debug messages.
> > Will mail it later.
> 
> Firmware hang seems to happen when REPLY_LEDS_CMD command is send.
> No idea if attached patch could help or not, but is worth to try it.
> It prevent to send LED command asynchronously, when possibly some
> other commands are pending in firmware.
> 
> If it does not help, please provide the same way captured debug
> messages from 2.6.38 kernel, where drivers works. I'll compare
> it with broken driver logs to figure out what broken driver
> do differently.
> 
> Thanks
> Stanislaw


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5677 bytes --]

Re: iwl4965: "MAC is in deep sleep!" freezes

[Jonathan Nieder]

Christoph Anton Mitterer wrote:

> Sorry for completely forgetting about this... (for so long)
>
> I've tried your patch now against 3.1, and it seems to not help.

Thanks.  Logs?  (If you can't get netconsole or a serial console
working, a photograph is fine.)

> Not sure whether trying with 2.6.38 helps.
> It "stopped" working within the very same kernel.

Not sure what you mean.  Are the quotation marks for emphasis, or do
you mean that when you replaced the mainboard it did not stop booting
in v2.6.38 but something else entirely?

As a reminder for forgetful people like me, the debugging patch in
question is at [1].

[1] http://thread.gmane.org/gmane.linux.kernel.wireless.general/76658/focus=1189537

Re: iwl4965: "MAC is in deep sleep!" freezes

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 1620 bytes --]

On Sun, 2011-11-13 at 19:46 -0600, Jonathan Nieder wrote:
> > I've tried your patch now against 3.1, and it seems to not help.
> 
> Thanks.  Logs?  (If you can't get netconsole or a serial console
> working, a photograph is fine.)

Which logs do you want,... then ones with the patch applied? Or without
and just from current 3.1?

(Anyways,... the error messages are just the same as before,... MAC is
in deep sleep, etc....)




> Not sure what you mean.  Are the quotation marks for emphasis, or do
> you mean that when you replaced the mainboard it did not stop booting
> in v2.6.38 but something else entirely?
Well...

I had a hardware problem with that notebook,.. it didn't boot anymore at
all.... got it to the vendor, and they replaced the whole mainboard
which had some hardware failure (including - I guess - the WiFi chip).
It reports to be the very same WiFi chipset.

The original problem was now resolved, but when I've used WLAN the first
time (and ever since than), I got the error messages as described in the
thread of [1] and [2].
Back then, I used the same kernel/same config/same build of it ... as I
had before the old mainboard broke,... and which worked perfectly fine.


I've used the double quotes,... as it didn't stop working completely. It
works.... but every now and then the system freezes for some ~30 seconds
during which it massively prints the "MAC is in deep sleep messages".


Thanks,
Chris.


[1] http://thread.gmane.org/gmane.linux.kernel.wireless.general/76658/focus=1189537
[2] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=636355

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5677 bytes --]

Re: iwl4965: "MAC is in deep sleep!" freezes

[Stanislaw Gruszka]

On Wed, Nov 16, 2011 at 07:08:53PM +0100, Christoph Anton Mitterer wrote:
> Which logs do you want,... then ones with the patch applied? Or without
Without patch

> and just from current 3.1?
Yes (I lose your old logs). But I also need verbose log for 2.6.38
(assuming 2.6.38 is the lates version where driver works for you)
to see what we do different regarding communication with firmware.
Capture good kernel log for about the same amout of time as bad
kernel log.

In summary: please provide logs as described previously here
http://article.gmane.org/gmane.linux.kernel/1187953
for 3.1 and 2.6.38.

Thanks
Stanislaw

Re: iwl4965: "MAC is in deep sleep!" freezes

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 765 bytes --]

Hi.


This took some time.

On Fri, 2011-11-18 at 08:45 +0100, Stanislaw Gruszka wrote:
> > and just from current 3.1?
> Yes (I lose your old logs). But I also need verbose log for 2.6.38
> (assuming 2.6.38 is the lates version where driver works for you)
> to see what we do different regarding communication with firmware.
I went back till 2.6.32, and the problem occurred with all kernels
again.

The logging is basically identical to what I've send before.

So either there is some weird hardware damage,.. or (as I suggested
before) it might be just some different revision or so of the chipset,
behaving different?!


Guess it's pointless to send you the same logging output again, but if
you'd like to I can do so.


Cheers,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5677 bytes --]

Re: iwl4965: "MAC is in deep sleep!" freezes

[Jonathan Nieder]

Christoph Anton Mitterer wrote:

> The logging is basically identical to what I've send before.
>
> So either there is some weird hardware damage,.. or (as I suggested
> before) it might be just some different revision or so of the chipset,
> behaving different?!
>
> Guess it's pointless to send you the same logging output again, but if
> you'd like to I can do so.

Yes, please do, or provide a link and summary so we don't have to hunt
it down.

Hope that helps,
Jonathan

Re: iwl4965: "MAC is in deep sleep!" freezes

[Stanislaw Gruszka]

On Thu, Dec 29, 2011 at 02:11:49PM -0600, Jonathan Nieder wrote:
> Christoph Anton Mitterer wrote:
> 
> > The logging is basically identical to what I've send before.
> >
> > So either there is some weird hardware damage,.. or (as I suggested
> > before) it might be just some different revision or so of the chipset,
> > behaving different?!

Hmm, on my experience, software sucks much more than hardware, but we
never can exclude hardware problem. I'm not sure if there are divergence
between iwl4965 chipsets. Also I'm not sure if there are more than one
chipsets revision at all. Christoph, you mentioned that you replaced you 
motherboard. Did the problem start to happen after that? Did you
correctly connect antennas cables to adapter. Does adapter sits
tight in the mini PCIe slot?

> > Guess it's pointless to send you the same logging output again, but if
> > you'd like to I can do so.
> 
> Yes, please do, or provide a link and summary so we don't have to hunt
> it down.

No, that's not needed if there are no good logs for compare. If possible
try on some very old kernel, first kernel with iwl4965 support was 2.6.24.

Thanks
Stanislaw

Re: iwl4965: "MAC is in deep sleep!" freezes

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 1185 bytes --]

On Tue, 2012-01-03 at 11:51 +0100, Stanislaw Gruszka wrote:
> Hmm, on my experience, software sucks much more than hardware, but we
> never can exclude hardware problem. I'm not sure if there are divergence
> between iwl4965 chipsets. Also I'm not sure if there are more than one
> chipsets revision at all.
Me neither unfortunately...

> Christoph, you mentioned that you replaced you 
> motherboard. Did the problem start to happen after that?
Yep,... the old board broke (and the system didn't bot anymore at
all)... it was replaced with a new one by the manufacturer... and
starting with that (with no software change on the disk in the meantime)
the wifi didn't work anymore (as described).


> Did you
> correctly connect antennas cables to adapter.
yep... at first (when they sent it back to me) it was not connected at
all... I connected it... the problem persists in both states, connected
and unconnected.

> Does adapter sits
> tight in the mini PCIe slot?
yep


> No, that's not needed if there are no good logs for compare. If possible
> try on some very old kernel, first kernel with iwl4965 support was 2.6.24.
I'll have a look...

Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5677 bytes --]

Re: iwl4965: "MAC is in deep sleep!" freezes

[Stanislaw Gruszka]

On Wed, Jan 04, 2012 at 02:22:28AM +0100, Christoph Anton Mitterer wrote:
> > Christoph, you mentioned that you replaced you 
> > motherboard. Did the problem start to happen after that?
> Yep,... the old board broke (and the system didn't bot anymore at
> all)... it was replaced with a new one by the manufacturer... and
> starting with that (with no software change on the disk in the meantime)
> the wifi didn't work anymore (as described).
Hmm, perhaps iwl4965 device does not work well with the new PCIe chipset.

Does any of fallowing workaround the problem?

kernel boot parameter:
pcie_aspm=off
pci_aspm=force
pci=nocrs
pci=use_crs

iwlegacy module parameter:
bt_coex_active=0

iwl4965 module parameters
11n_disable=1

> > Did you
> > correctly connect antennas cables to adapter.
> yep... at first (when they sent it back to me) it was not connected at
> all... I connected it... the problem persists in both states, connected
> and unconnected.
There should be 3 or 2 antennas: 1 - white, 3 - grey, 2 - black,
connection on adapter should match cable color (ok, I don't know if
it does any difference, but it worth to try). If there is only one
antenna available, perhaps there is somewhere another miniPCIe slot
where are more antenna cables available. But, if there is really
only one antenna cable, it should be connected to number 1 (white)
adapter connector.

Stanislaw

Re: iwl4965: "MAC is in deep sleep!" freezes

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 1301 bytes --]

On Thu, 2012-01-05 at 11:14 +0100, Stanislaw Gruszka wrote:
> Hmm, perhaps iwl4965 device does not work well with the new PCIe chipset.
> 
> Does any of fallowing workaround the problem?
> 
> kernel boot parameter:
> pcie_aspm=off
> pci_aspm=force
> pci=nocrs
> pci=use_crs
> 
> iwlegacy module parameter:
> bt_coex_active=0
> 
> iwl4965 module parameters
> 11n_disable=1

None of these helped,.. it seems however that even with pci_aspm=force
it:
ACPI _OSC control for PCIe not granted, disabling ASPM


> There should be 3 or 2 antennas: 1 - white, 3 - grey, 2 - black,
> connection on adapter should match cable color (ok, I don't know if
> it does any difference, but it worth to try). If there is only one
> antenna available, perhaps there is somewhere another miniPCIe slot
> where are more antenna cables available. But, if there is really
> only one antenna cable, it should be connected to number 1 (white)
> adapter connector.
Well what I thought to be the wifi controller till now, has only on
cable (white).... with only one connector.

I tried to disassemble the whole thing but must shamefully admit that I
wasn't able to... at some place it's kept somehow together (als screws
are loosened) and I couldn't find out so far how.



Cheers,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5677 bytes --]