[BUG] Suspend tries to suspend devices already in D3

[BUG] Suspend tries to suspend devices already in D3

[Frans Pop]

This is the second time I've hit this warning on an STR. Most times
I do *not* get the warning. The first line is an extra debug printk I added
after the first occurrence to give me the driver causing the problem.

What is different is that before this suspend I had switched to my wired
e1000e NIC and had done an ifdown on the wireless. I suspect that because
of that the wireless already was at D3 _before_ the suspend was started.

Could this be the same issue as reported by Ming Lei?

Kernel is v2.6.29-rc3-100-gf2257b7 with the following relevant extra patches:
- PCI PM: Fix suspend error paths and testing facility breakage
- ricoh_mmc: Use suspend_late/resume_early

Cheers,
FJP

kernel: iwlagn 0000:10:00.0: has incorrect power state
kernel: ------------[ cut here ]------------
kernel: WARNING: at drivers/pci/pci-driver.c:375 pci_legacy_suspend+0xd4/0x119()
kernel: Hardware name: HP Compaq 2510p Notebook PC
kernel: Modules linked in: tcp_diag inet_diag i915 drm i2c_algo_bit i2c_core ppdev parport_pc lp parport 
ipv6 nfs lockd nfs_acl auth_rpcgss sunrpc ext2 coretemp hp_wmi acpi_cpufreq loop snd_hda_codec_analog arc4 
snd_hda_intel ecb snd_hda_codec snd_pcm_oss snd_mixer_oss iwlagn iwlcore snd_pcm rfkill snd_seq_dummy 
lib80211 mac80211 pcmcia snd_seq_oss snd_seq_midi snd_rawmidi cfg80211 snd_seq_midi_event snd_seq snd_timer 
snd_seq_device yenta_socket rsrc_nonstatic pcmcia_core serio_raw iTCO_wdt psmouse pcspkr usblp snd soundcore 
snd_page_alloc wmi hp_accel intel_agp video output battery lis3lv02d container ac button joydev evdev ext3 
jbd mbcache sha256_generic aes_x86_64 aes_generic cbc usbhid hid dm_crypt dm_mirror dm_region_hash dm_log 
dm_snapshot dm_mod sg sr_mod cdrom sd_mod piix ata_piix ide_pci_generic ide_core pata_acpi ohci1394 
sdhci_pci sdhci ricoh_mmc ieee1394 mmc_core led_class ata_generic uhci_hcd ehci_hcd libata e1000e scsi_mod 
thermal processor fan thermal_sys
kernel: Pid: 14203, comm: s2ram Not tainted 2.6.29-rc3 #13
kernel: Call Trace:
acpid: client 13573[0:0] has disconnected
acpid: client 4290[0:0] has disconnected
kernel:  [<ffffffff8023995b>] warn_slowpath+0xd3/0x10f
kernel:  [<ffffffff8024fc19>] ? down_trylock+0x2d/0x34
kernel:  [<ffffffff80446562>] ? _spin_unlock+0x9/0xb
kernel:  [<ffffffff80443ff6>] ? printk+0x67/0x69
kernel:  [<ffffffff803649c0>] ? acpi_bus_power_manageable+0x11/0x29
kernel:  [<ffffffff8033771f>] ? acpi_pci_power_manageable+0x17/0x19
kernel:  [<ffffffff8032ef75>] ? pci_set_power_state+0xcc/0x101
kernel:  [<ffffffff803313dd>] pci_legacy_suspend+0xd4/0x119
kernel:  [<ffffffff80331515>] pci_pm_suspend+0x34/0x86
kernel:  [<ffffffff80398e0e>] pm_op+0x52/0xe5
kernel:  [<ffffffff803993b8>] device_suspend+0x32e/0x453
kernel:  [<ffffffff8025f4a6>] suspend_devices_and_enter+0x3e/0x12e
kernel:  [<ffffffff8025f6fe>] enter_state+0x110/0x164
kernel:  [<ffffffff8025f809>] state_store+0xb7/0xd7
kernel:  [<ffffffff8032014f>] kobj_attr_store+0x17/0x19
kernel:  [<ffffffff802e1991>] sysfs_write_file+0xdf/0x114
kernel:  [<ffffffff8029729f>] vfs_write+0xa7/0xe1
kernel:  [<ffffffff8029739d>] sys_write+0x47/0x6f
kernel:  [<ffffffff8020c25b>] system_call_fastpath+0x16/0x1b
kernel: ---[ end trace 7c1037001dd4a99a ]---

Re: [BUG] Suspend tries to suspend devices already in D3

[Rafael J. Wysocki]

On Tuesday 03 February 2009, Frans Pop wrote:
> This is the second time I've hit this warning on an STR. Most times
> I do *not* get the warning. The first line is an extra debug printk I added
> after the first occurrence to give me the driver causing the problem.
> 
> What is different is that before this suspend I had switched to my wired
> e1000e NIC and had done an ifdown on the wireless. I suspect that because
> of that the wireless already was at D3 _before_ the suspend was started.

If ifdown causes the device to go into D3, then this is possible.

> Could this be the same issue as reported by Ming Lei?

Yes, it could.

Thanks,
Rafael

Re: [BUG] Suspend tries to suspend devices already in D3

[Frans Pop]

On Tuesday 03 February 2009, Rafael J. Wysocki wrote:
> > What is different is that before this suspend I had switched to my
> > wired e1000e NIC and had done an ifdown on the wireless. I suspect
> > that because of that the wireless already was at D3 _before_ the
> > suspend was started.
>
> If ifdown causes the device to go into D3, then this is possible.

Note that I'm not 100% sure the device was in D3, but it seems likely.
All we really know is that it wasn't in D0.

Re: [BUG] Suspend tries to suspend devices already in D3

[Frans Pop]

On Tuesday 03 February 2009, you wrote:
> On Tuesday 03 February 2009, Frans Pop wrote:
> > This is the second time I've hit this warning on an STR. Most times
> > I do *not* get the warning. The first line is an extra debug printk I
> > added after the first occurrence to give me the driver causing the
> > problem.
> >
> > What is different is that before this suspend I had switched to my
> > wired e1000e NIC and had done an ifdown on the wireless. I suspect
> > that because of that the wireless already was at D3 _before_ the
> > suspend was started.

The warning is reproducible if wlan0 is down during suspend.

> If ifdown causes the device to go into D3, then this is possible.

I've done some extra checking and if I read it correctly lspci -vvv lists
the status as D0 after ifdown, so that's probably not it. I've also added
extra info in the warning and at that time the status is D3hot.

Looking at iwl-agn.c the cause could be that iwl_pci_suspend() depends on
iwl_mac_stop() to call pci_save_state(). I added some debug statements in 
iwl_pci_suspend() that showed priv->is_open is false, so iwl_mac_stop 
does indeed *not* get called when the interface is down.

Is that it? Not sure what the correct fix would be if it is.

Cheers,
FJP

Re: [BUG] Suspend tries to suspend devices already in D3

[Frans Pop]

On Tuesday 03 February 2009, Frans Pop wrote:
> Looking at iwl-agn.c the cause could be that iwl_pci_suspend() depends
> on iwl_mac_stop() to call pci_save_state(). I added some debug
> statements in iwl_pci_suspend() that showed priv->is_open is false, so
> iwl_mac_stop does indeed *not* get called when the interface is down.

However, iwl_mac_stop() had been called when I ran ifdown just before
suspending the system:

# Here I run 'ifdown wlan0':
Feb  3 18:25:30 aragorn kernel: wlan0: deauthenticating by local choice (reason=3)
Feb  3 18:25:30 aragorn kernel: iwlagn: MAC is in deep sleep!
# The next debug statement is from iwl_mac_stop():
Feb  3 18:25:30 aragorn kernel: iwlagn FJP: state saved and device disabled
Feb  3 18:25:30 aragorn kernel: wlan0: deauthenticating by local choice (reason=3)
# Start of suspend:
Feb  3 18:25:54 aragorn kernel: PM: Syncing filesystems ... done.
Feb  3 18:25:54 aragorn kernel: Freezing user space processes ... (elapsed 0.00 seconds) done.
Feb  3 18:25:54 aragorn kernel: Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
Feb  3 18:25:54 aragorn kernel: Suspending console(s) (use no_console_suspend to debug)
Feb  3 18:25:54 aragorn kernel: sd 0:0:0:0: [sda] Synchronizing SCSI cache
Feb  3 18:25:54 aragorn kernel: sd 0:0:0:0: [sda] Stopping disk
Feb  3 18:25:54 aragorn kernel: ACPI handle has no context!
Feb  3 18:25:54 aragorn kernel: ACPI handle has no context!
Feb  3 18:25:54 aragorn kernel: sdhci-pci 0000:02:06.2: PME# disabled
Feb  3 18:25:54 aragorn kernel: ACPI handle has no context!
Feb  3 18:25:54 aragorn acpid: client 4081[0:0] has disconnected
Feb  3 18:25:54 aragorn kernel: ACPI handle has no context!
# The next debug statement is from iwl_pci_suspend() and shows priv->is_open is false:
Feb  3 18:25:54 aragorn kernel: iwlagn FJP: not calling iwl_mac_stop
Feb  3 18:25:54 aragorn kernel: ------------[ cut here ]------------
Feb  3 18:25:54 aragorn kernel: WARNING: at drivers/pci/pci-driver.c:375 pci_legacy_suspend+0xd4/0x11a()
Feb  3 18:25:54 aragorn kernel: Hardware name: HP Compaq 2510p Notebook PC
Feb  3 18:25:54 aragorn kernel: PCI PM: Device state not saved by iwl_pci_suspend+0x0/0x6a [iwlagn], state is D3hot
[...]

Re: [BUG] Suspend tries to suspend devices already in D3

[Rafael J. Wysocki]

On Tuesday 03 February 2009, Frans Pop wrote:
> On Tuesday 03 February 2009, Frans Pop wrote:
> > Looking at iwl-agn.c the cause could be that iwl_pci_suspend() depends
> > on iwl_mac_stop() to call pci_save_state(). I added some debug
> > statements in iwl_pci_suspend() that showed priv->is_open is false, so
> > iwl_mac_stop does indeed *not* get called when the interface is down.
> 
> However, iwl_mac_stop() had been called when I ran ifdown just before
> suspending the system:

The problem that the warning can be triggered by a device being in a low
power state already before suspend should be fixed by the PCI PM fixes I've
just posted.

Thanks,
Rafael

Re: [BUG] Suspend tries to suspend devices already in D3

[Frans Pop]

On Wednesday 04 February 2009, Rafael J. Wysocki wrote:
> On Tuesday 03 February 2009, Frans Pop wrote:
> > On Tuesday 03 February 2009, Frans Pop wrote:
> > > Looking at iwl-agn.c the cause could be that iwl_pci_suspend()
> > > depends on iwl_mac_stop() to call pci_save_state(). I added some
> > > debug statements in iwl_pci_suspend() that showed priv->is_open is
> > > false, so iwl_mac_stop does indeed *not* get called when the
> > > interface is down.
> >
> > However, iwl_mac_stop() had been called when I ran ifdown just before
> > suspending the system:
>
> The problem that the warning can be triggered by a device being in a
> low power state already before suspend should be fixed by the PCI PM
> fixes I've just posted.

I'm afraid I still get the warning with your latest patches on top of 
v2.6.29-rc3-495-gdcf6a79.

IIUC the device is not yet in low power state when I suspend, but only 
disabled. The subject of the mail proved to be incorrect, sorry.

Please take a look at the functions from iwl-agn.c mentioned above. I 
think the problem is there.

Cheers,
FJP

Re: [BUG] Suspend tries to suspend devices already in D3

[Rafael J. Wysocki]

On Wednesday 04 February 2009, Frans Pop wrote:
> On Wednesday 04 February 2009, Rafael J. Wysocki wrote:
> > On Tuesday 03 February 2009, Frans Pop wrote:
> > > On Tuesday 03 February 2009, Frans Pop wrote:
> > > > Looking at iwl-agn.c the cause could be that iwl_pci_suspend()
> > > > depends on iwl_mac_stop() to call pci_save_state(). I added some
> > > > debug statements in iwl_pci_suspend() that showed priv->is_open is
> > > > false, so iwl_mac_stop does indeed *not* get called when the
> > > > interface is down.
> > >
> > > However, iwl_mac_stop() had been called when I ran ifdown just before
> > > suspending the system:
> >
> > The problem that the warning can be triggered by a device being in a
> > low power state already before suspend should be fixed by the PCI PM
> > fixes I've just posted.
> 
> I'm afraid I still get the warning with your latest patches on top of 
> v2.6.29-rc3-495-gdcf6a79.

If you're still getting the warning with these patches, then yes, the problem
appears to be in iwl-agn.c .
 
> IIUC the device is not yet in low power state when I suspend, but only 
> disabled. The subject of the mail proved to be incorrect, sorry.
> 
> Please take a look at the functions from iwl-agn.c mentioned above. I 
> think the problem is there.

I will.

Thanks,
Rafael

Re: [BUG] Suspend tries to suspend devices already in D3

[Frans Pop]

On Wednesday 04 February 2009, Rafael J. Wysocki wrote:
> On Wednesday 04 February 2009, Frans Pop wrote:
> > On Wednesday 04 February 2009, Rafael J. Wysocki wrote:
> > > On Tuesday 03 February 2009, Frans Pop wrote:
> > > > On Tuesday 03 February 2009, Frans Pop wrote:
> > > > > Looking at iwl-agn.c the cause could be that iwl_pci_suspend()
> > > > > depends on iwl_mac_stop() to call pci_save_state(). I added
> > > > > some debug statements in iwl_pci_suspend() that showed
> > > > > priv->is_open is false, so iwl_mac_stop does indeed *not* get
> > > > > called when the interface is down.
> > > >
> > > > However, iwl_mac_stop() had been called when I ran ifdown just
> > > > before suspending the system:
> > >
> > > The problem that the warning can be triggered by a device being in
> > > a low power state already before suspend should be fixed by the PCI
> > > PM fixes I've just posted.
> >
> > I'm afraid I still get the warning with your latest patches on top of
> > v2.6.29-rc3-495-gdcf6a79.
>
> If you're still getting the warning with these patches, then yes, the
> problem appears to be in iwl-agn.c .

FYI, this looks to be fixed in -rc5 by c4e061ace755 + 89c581b3fb29.

Thanks,
FJP