Linux kernel 2.6.17-git14 and PCI suspend/resume

Linux kernel 2.6.17-git14 and PCI suspend/resume

[Lukas Hejtmanek]

Hello,

in this kernel I'm seeing these messages after S3 resume:
kernel: PM: Writing back config space on device 0000:01:01.1 at offset f (was 4020205, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset e (was 0, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset d (was dc, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset c (was 0, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset b (was 19671043, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset a (was 0, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 9 (was 0, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 8 (was 0, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 7 (was 0, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 6 (was 0, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 5 (was 0, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 4 (was fe8fd800, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 3 (was 804000, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 2 (was c001008, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 1 (was 2100006, writing ffffffff)
kernel: PM: Writing back config space on device 0000:01:01.1 at offset 0 (was 5521180, writing ffffffff)

Which actually mess up PCI config space (and sdhci driver is unable to set up
MMC device correctly). Do you have any idea what to try?

These messages appear for devices that are not handled by any loaded module.

-- 
Lukáš Hejtmánek

Re: Linux kernel 2.6.17-git14 and PCI suspend/resume

[Greg KH]

On Tue, Jul 04, 2006 at 01:11:21AM +0200, Lukas Hejtmanek wrote:
> Hello,
> 
> in this kernel I'm seeing these messages after S3 resume:
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset f (was 4020205, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset e (was 0, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset d (was dc, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset c (was 0, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset b (was 19671043, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset a (was 0, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 9 (was 0, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 8 (was 0, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 7 (was 0, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 6 (was 0, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 5 (was 0, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 4 (was fe8fd800, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 3 (was 804000, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 2 (was c001008, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 1 (was 2100006, writing ffffffff)
> kernel: PM: Writing back config space on device 0000:01:01.1 at offset 0 (was 5521180, writing ffffffff)
> 
> Which actually mess up PCI config space (and sdhci driver is unable to set up
> MMC device correctly). Do you have any idea what to try?

When suspending, pci_save_state() would have saved off those values (all
1s) which is what it is restoring.  That function gets called if there
is no driver specific suspend function to call.  On suspend, is there
any driver loaded for the device?

And what type of PCI device is this?

> These messages appear for devices that are not handled by any loaded module.

Ick.

Any chance you can test -mm and see if that helps with suspend/resume?
Linus's new suspend framework is in there, and that's the way forward
for these kinds of issues.

thanks,

greg k-h

Re: Linux kernel 2.6.17-git14 and PCI suspend/resume

[Lukas Hejtmanek]

On Mon, Jul 03, 2006 at 04:21:09PM -0700, Greg KH wrote:
> When suspending, pci_save_state() would have saved off those values (all
> 1s) which is what it is restoring.  That function gets called if there
> is no driver specific suspend function to call.  On suspend, is there
> any driver loaded for the device?

No driver is loaded.

> And what type of PCI device is this?

It does for these devices:
01:01.0 CardBus bridge: Ricoh Co Ltd RL5c476 II (rev b3)
01:01.1 FireWire (IEEE 1394): Ricoh Co Ltd R5C552 IEEE 1394 Controller (rev 08)
01:01.2 Class 0805: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev
17)
01:01.3 System peripheral: Ricoh Co Ltd R5C592 Memory Stick Bus Host Adapter
(rev 08)
01:01.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev 03)


> Any chance you can test -mm and see if that helps with suspend/resume?
> Linus's new suspend framework is in there, and that's the way forward
> for these kinds of issues.

I found out, that pci_save_state() reads 0xffffffff value using
pci_read_config_dword for these devices. Actually, I modified pci_save_state
like this:
int
pci_save_state(struct pci_dev *dev)
{
        int i;
        /* XXX: 100% dword access ok here? */
        printk(KERN_ERR "PM: Saving config space on device %s.\n",
                        pci_name(dev));
        mdelay(500);
        for (i = 0; i < 16; i++) {
                pci_read_config_dword(dev, i * 4,&dev->saved_config_space[i]);
                if(strcmp(pci_name(dev), "0000:01:01.4") == 0)
                        printk(KERN_ERR "PM: Saved value: %x.\n", dev->saved_config_space[i]);
        }
        if ((i = pci_save_msi_state(dev)) != 0)
                return i;
        if ((i = pci_save_msix_state(dev)) != 0)
                return i;
        return 0;
}

With this result in log:
kernel: PM: Saving config space on device 0000:01:01.4.
kernel: PM: Saved value: ffffffff.
last message repeated 15 times

So, looks like not enabled/mapped device returns 1s. 
(For none of mentioned devices is loaded driver, so they are all disabled).

So for now, I disabled to restore state of value 0xffffffff and things go OK.

-- 
Lukáš Hejtmánek

Re: Linux kernel 2.6.17-git14 and PCI suspend/resume

[Lukas Hejtmanek]

Hello,

On Mon, Jul 03, 2006 at 04:21:09PM -0700, Greg KH wrote:
> When suspending, pci_save_state() would have saved off those values (all
> 1s) which is what it is restoring.  That function gets called if there
> is no driver specific suspend function to call.  On suspend, is there
> any driver loaded for the device?

I was playing a little with various versions to see what has changed.
It looks that somewhere after 2.6.17-rc6 just printk message was added and
pci_save/restore_state newer worked for me.

The strange is that only some of devices are unable to save pci config.

According to lspci -t my PCI tree looks like this:

(Devices marked with * do not save config correctly. The others are OK. None of marked devices is claimed by any driver in kernel, as well as IPW2915ABG device is not claimed by any driver, but this device is restored correctly.)

# lspci -t
-[0000:00]-+-00.0
           +-01.0-[0000:03]--
           +-02.0
           +-02.1
           +-1b.0
           +-1d.0
           +-1d.1
           +-1d.2
           +-1d.3
           +-1d.7
           +-1e.0-[0000:01-02]--+-00.0
           |                    +-01.0 *
           |                    +-01.1 *
           |                    +-01.2 *
           |                    +-01.3 *
           |                    +-01.4 *
           |                    \-02.0
           +-1f.0
           \-1f.1

This is plain lspci:
00:00.0 Host bridge: Intel Corporation Mobile 915GM/PM/GMS/910GML Express Processor to DRAM Controller (rev 03)
00:01.0 PCI bridge: Intel Corporation Mobile 915GM/PM Express PCI Express Root Port (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile 915GM/GMS/910GML Express Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation Mobile 915GM/GMS/910GML Express Gr aphics Controller (rev 03)
00:1b.0 Audio device: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) High Definition Audio Controller (rev 04)
00:1d.0 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) US B UHCI #1 (rev 04)
00:1d.1 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) US B UHCI #2 (rev 04)
00:1d.2 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) US B UHCI #3 (rev 04)
00:1d.3 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) US B UHCI #4 (rev 04)
00:1d.7 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) US B2 EHCI Controller (rev 04)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev d4)
00:1f.0 ISA bridge: Intel Corporation 82801FBM (ICH6M) LPC Interface Bridge (rev 04)
00:1f.1 IDE interface: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) IDE Controller (rev 04)
01:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller (rev 13)
01:01.0 CardBus bridge: Ricoh Co Ltd RL5c476 II (rev b3)
01:01.1 FireWire (IEEE 1394): Ricoh Co Ltd R5C552 IEEE 1394 Controller (rev 08)
01:01.2 Class 0805: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev 17)
01:01.3 System peripheral: Ricoh Co Ltd R5C592 Memory Stick Bus Host Adapter (rev 08)
01:01.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev 03)
01:02.0 Network controller: Intel Corporation PRO/Wireless 2915ABG Network Connection (rev 05)


-- 
Lukáš Hejtmánek