[PATCH -v5 0/6] Use PCI Serial Number to identify device change

[PATCH -v5 0/6] Use PCI Serial Number to identify device change

[Yijing Wang]

v4->v5: update patch 6/6, correct some mistakes in this patch, thank Paul for help
		test this series.
v3->v4: introudce PCI VPD Sn support suggested by Bjorn, and other rework things.
		
v2->v3: add a wrap function pci_dsn_init, fix other typo error and git am error.
		Thanks for Don Dutile and Paul Bolle 's review,comments and test.
v1->v2: Modify pci_get_dsn to pci_device_serial_number,
	    power off slot before remove the old device during resume to avoid
		old .remove() method to touch new hardware.
		Fix other typo and fail check problems.
		split the list_empty() guard into new patch.
	    Thanks for Bjorn's review and comments.

I tested to apply this series to Linus 3.11-rc4 and Bjorn's pci-next branch, and
result is ok.

My test machine arch is IA64, it's the only one support pciehp.
But IA64 default does not support SUSPEND. So I modified my kernel to run pm suspend and resume test.

-+-[0000:40]-+-00.0-[0000:41]--
...............[snip]..............
 |           +-05.0-[0000:45]--
 |           +-07.0-[0000:46]--+-00.0  Intel Corporation 82576 Gigabit Network Connection
 |           |                 \-00.1  Intel Corporation 82576 Gigabit Network Connection

dmesg info after suspend->resume test:
......................................
pciehp 0000:40:03.0:pcie04: HPC vendor_id 8086 device_id 340a ss_vid 8086 ss_did 340a
pciehp 0000:40:03.0:pcie04: service driver pciehp loaded
pciehp 0000:40:04.0:pcie04: HPC vendor_id 8086 device_id 340b ss_vid 8086 ss_did 340b
pciehp 0000:40:04.0:pcie04: service driver pciehp loaded
pciehp 0000:40:05.0:pcie04: HPC vendor_id 8086 device_id 340c ss_vid 8086 ss_did 340c
pciehp 0000:40:05.0:pcie04: service driver pciehp loaded
pciehp 0000:40:07.0:pcie04: HPC vendor_id 8086 device_id 340e ss_vid 8086 ss_did 340e
pciehp 0000:40:07.0:pcie04: service driver pciehp loaded
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
PM: Syncing filesystems ... done.
PM: Preparing system for mem sleep
Freezing user space processes ... (elapsed 0.003 seconds) done.
Freezing remaining freezable tasks ... (elapsed 0.004 seconds) done.
PM: Entering mem sleep
Suspending console(s) (use no_console_suspend to debug)
sd 5:0:0:0: [sdb] Synchronizing SCSI cache
mptbase: ioc1: pci-suspend: pdev=0xe000001f03611600, slot=0000:43:00.0, Entering operating state [D3]
mptbase: ioc0: pci-suspend: pdev=0xe000001f035de500, slot=0000:02:00.0, Entering operating state [D3]
pcieport 0000:40:07.0: System wakeup enabled by ACPI
GSI 61 (level, low) -> CPU 28 (0x3400) vector 128 unregistered
pcieport 0000:40:07.0: System wakeup enabled by ACPI
GSI 64 (level, low) -> CPU 25 (0x3100) vector 109 unregistered
GSI 40 (level, low) -> CPU 23 (0x2700) vector 89 unregistered
GSI 37 (level, low) -> CPU 12 (0x1400) vector 79 unregistered
PM: suspend of devices complete after 444.758 msecs
suspend debug: Waiting for 5 seconds.
pciehp_resume: bus 0x43, coming here!
usb usb3: root hub lost power or was reset
pciehp_resume: bus 0x44, coming here!
pciehp_resume: bus 0x45, coming here!
usb usb2: root hub lost power or was reset
usb usb1: root hub lost power or was reset
usb usb4: root hub lost power or was reset
usb usb5: root hub lost power or was reset
usb usb6: root hub lost power or was reset
pciehp_resume: bus 0x46, coming here!
GSI 37 (level, low) -> CPU 29 (0x3500) vector 79
mptbase: ioc0: pci-resume: pdev=0xe000001f035de500, slot=0000:02:00.0, Previous operating state [D0]
GSI 40 (level, low) -> CPU 30 (0x3600) vector 86
mptbase: ioc0: pci-resume: ioc-state=0x1,doorbell=0x10000000
GSI 64 (level, low) -> CPU 31 (0x3700) vector 101
ata4: SATA link down (SStatus 0 SControl 300)
ata3: SATA link down (SStatus 0 SControl 300)
usb 3-2: reset low-speed USB device number 3 using uhci_hcd
ata1.00: SATA link down (SStatus 0 SControl 300)
ata1.01: SATA link down (SStatus 0 SControl 300)
ata2.00: SATA link down (SStatus 0 SControl 300)
ata2.01: SATA link down (SStatus 0 SControl 300)
usb 4-2: reset full-speed USB device number 3 using uhci_hcd
pciehp_resume: status 0
pciehp_resume: status 1
pciehp_resume: status 0
pciehp_resume: status 1
pciehp_resume: device 0000:46:00.0, serial number changed!
mptbase: ioc1: pci-resume: pdev=0xe000001f03611600, slot=0000:43:00.0, Previous operating state [D0]
mptbase: ioc1: pci-resume: ioc-state=0x1,doorbell=0x10000000
atkbd serio0: keyboard reset failed on isa0060/serio0
atkbd serio1: keyboard reset failed on isa0060/serio1
igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
pciehp_unconfigure_device: device 0000:46:00.0, preparing remove!
igb 0000:46:00.0: removed PHC on eth17    ------------------------------->remove old device, because I change its PCIe DSN in pciehp_resume() for test situation which device serial number changed.
pciehp_unconfigure_device: device 0000:46:00.1, preparing remove!
igb 0000:46:00.1: removed PHC on eth16
pciehp_resume: remove device complete!
pci 0000:46:00.0: [8086:10c9] type 00 class 0x020000
pci 0000:46:00.0: reg 0x10: [mem 0x00000000-0x0001ffff]
pci 0000:46:00.0: reg 0x14: [mem 0x00000000-0x0001ffff]
pci 0000:46:00.0: reg 0x18: [io  0x0000-0x001f]
pci 0000:46:00.0: reg 0x1c: [mem 0x00000000-0x00003fff]
pci 0000:46:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
pci 0000:46:00.0: PME# supported from D0 D3hot D3cold
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.1: [8086:10c9] type 00 class 0x020000
pci 0000:46:00.1: reg 0x10: [mem 0x00000000-0x0001ffff]
pci 0000:46:00.1: reg 0x14: [mem 0x00000000-0x0001ffff]
pci 0000:46:00.1: reg 0x18: [io  0x0000-0x001f]
pci 0000:46:00.1: reg 0x1c: [mem 0x00000000-0x00003fff]
pci 0000:46:00.1: reg 0x30: [mem 0x00000000-0x0001ffff pref]
pci 0000:46:00.1: PME# supported from D0 D3hot D3cold
pci 0000:46:00.1: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.1: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.1: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.1: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: BAR 0: assigned [mem 0x80000000-0x8001ffff]
pci 0000:46:00.0: BAR 1: assigned [mem 0x80020000-0x8003ffff]
pci 0000:46:00.0: BAR 6: assigned [mem 0x80700000-0x8071ffff pref]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: BAR 10: can't assign mem (size 0x100000)
pci 0000:46:00.1: BAR 0: assigned [mem 0x80040000-0x8005ffff]
pci 0000:46:00.1: BAR 1: assigned [mem 0x80060000-0x8007ffff]
pci 0000:46:00.1: BAR 6: assigned [mem 0x80720000-0x8073ffff pref]
pci 0000:46:00.1: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.1: BAR 10: can't assign mem (size 0x100000)
pci 0000:46:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: BAR 7: assigned [mem 0x80080000-0x800fffff 64bit]
pci 0000:46:00.1: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.1: BAR 7: can't assign mem (size 0x80000)
pci 0000:46:00.0: BAR 3: can't assign mem (size 0x4000)
pci 0000:46:00.1: BAR 3: can't assign mem (size 0x4000)
pci 0000:46:00.0: BAR 2: assigned [io  0x9000-0x901f]
pci 0000:46:00.1: BAR 2: assigned [io  0x9020-0x903f]
pcieport 0000:40:07.0: PCI bridge to [bus 46]
pcieport 0000:40:07.0:   bridge window [io  0x9000-0xafff]
pcieport 0000:40:07.0:   bridge window [mem 0x80000000-0x800fffff]
pcieport 0000:40:07.0:   bridge window [mem 0x80700000-0x809fffff 64bit pref]
PCI: No. 2 try to assign unassigned res
release child resource [mem 0x80000000-0x8001ffff]
release child resource [mem 0x80020000-0x8003ffff]
release child resource [mem 0x80040000-0x8005ffff]
release child resource [mem 0x80060000-0x8007ffff]
release child resource [mem 0x80080000-0x800fffff 64bit]
pcieport 0000:40:07.0: resource 14 [mem 0x80000000-0x800fffff] released
pcieport 0000:40:07.0: PCI bridge to [bus 46]
pcieport 0000:40:07.0: bridge window [mem 0x00100000-0x001fffff] to [bus 46] add_size 400000
pcieport 0000:40:07.0: res[14]=[mem 0x00100000-0x001fffff] get_res_add_size add_size 400000
pcieport 0000:40:07.0: BAR 14: assigned [mem 0x81600000-0x81afffff]
pci 0000:46:00.0: reg 0x184: [mem 0x80080000-0x8008ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x80080000-0x8008ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x80080000-0x8008ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x80080000-0x8008ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x80080000-0x8008ffff 64bit]
pci 0000:46:00.1: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x80080000-0x8008ffff 64bit]
pci 0000:46:00.1: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x80080000-0x8008ffff 64bit]
pci 0000:46:00.0: res[10]=[mem 0x00000000-0xffffffffffffffff 64bit] get_res_add_size add_size 100000
pci 0000:46:00.1: res[10]=[mem 0x00000000-0xffffffffffffffff 64bit] get_res_add_size add_size 100000
pci 0000:46:00.0: res[7]=[mem 0x00000000-0xffffffffffffffff 64bit] get_res_add_size add_size 80000
pci 0000:46:00.1: res[7]=[mem 0x00000000-0xffffffffffffffff 64bit] get_res_add_size add_size 80000
pci 0000:46:00.0: BAR 0: assigned [mem 0x81600000-0x8161ffff]
pci 0000:46:00.0: BAR 1: assigned [mem 0x81620000-0x8163ffff]
pci 0000:46:00.0: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.0: BAR 10: assigned [mem 0x81640000-0x8173ffff 64bit]
pci 0000:46:00.1: BAR 0: assigned [mem 0x81740000-0x8175ffff]
pci 0000:46:00.1: BAR 1: assigned [mem 0x81760000-0x8177ffff]
pci 0000:46:00.1: reg 0x190: [mem 0x00000000-0x0001ffff 64bit]
pci 0000:46:00.1: BAR 10: assigned [mem 0x81780000-0x8187ffff 64bit]
pci 0000:46:00.0: reg 0x184: [mem 0x80080000-0x8008ffff 64bit]
pci 0000:46:00.0: BAR 7: assigned [mem 0x81880000-0x818fffff 64bit]
pci 0000:46:00.1: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0000:46:00.1: BAR 7: assigned [mem 0x81900000-0x8197ffff 64bit]
pci 0000:46:00.0: BAR 3: assigned [mem 0x81980000-0x81983fff]
pci 0000:46:00.1: BAR 3: assigned [mem 0x81984000-0x81987fff]
pcieport 0000:40:07.0: PCI bridge to [bus 46]
pcieport 0000:40:07.0:   bridge window [io  0x9000-0xafff]
pcieport 0000:40:07.0:   bridge window [mem 0x81600000-0x81afffff]
pcieport 0000:40:07.0:   bridge window [mem 0x80700000-0x809fffff 64bit pref]
igb 0000:46:00.0: enabling device (0100 -> 0102)
igb 0000:46:00.0: added PHC on eth4
igb 0000:46:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:46:00.0: eth4: (PCIe:2.5Gb/s:Width x4) 00:0e:0c:ff:ff:ff
igb 0000:46:00.0: eth4: PBA No: FFFFFF-0FF
igb 0000:46:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
igb 0000:46:00.1: enabling device (0100 -> 0102)
GSI 61 (level, low) -> CPU 0 (0x1000) vector 128
igb 0000:46:00.1: added PHC on eth5
igb 0000:46:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:46:00.1: eth5: (PCIe:2.5Gb/s:Width x4) 00:0e:0c:ff:ff:fe
igb 0000:46:00.1: eth5: PBA No: FFFFFF-0FF
igb 0000:46:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
mptbase: ioc1: Sending mpt_do_ioc_recovery
mptbase: ioc1: Initiating bringup
mptbase: ioc0: Sending mpt_do_ioc_recovery
mptbase: ioc0: Initiating bringup
ioc1: LSISAS1064E B2: Capabilities={Initiator}
ioc0: LSISAS1064E B2: Capabilities={Initiator}
pcieport 0000:40:07.0: System wakeup disabled by ACPI
pcieport 0000:40:07.0: System wakeup disabled by ACPI
mptbase: ioc1: pci-resume: success
mptbase: ioc0: pci-resume: success
PM: resume of devices complete after 17542.954 msecs
PM: Finishing wakeup.
Restarting tasks ... done.


Yijing Wang (6):
  PCI,pciehp: avoid add a device already exist before suspend during
    resume
  PCI: introduce PCIe Device Serial Number Capability support
  PCI: Introduce Vital Product Data Serial Number capability support
  PCI: add pci_serial_number_changed() for device change identification
  PCI: add inspection of device change in pci_scan_single_device
  PCI,pciehp: identify device change during suspend

 drivers/pci/access.c              |   13 +-----
 drivers/pci/hotplug/pciehp_core.c |   37 ++++++++++++++++-
 drivers/pci/pci.c                 |   77 +++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.h                 |   14 ++++++-
 drivers/pci/probe.c               |    4 ++
 drivers/pci/vpd.c                 |   70 +++++++++++++++++++++++++++++++++
 include/linux/pci.h               |    4 ++
 7 files changed, 205 insertions(+), 14 deletions(-)

[PATCH -v5 1/6] PCI,pciehp: avoid add a device already exist before suspend during resume

[Yijing Wang]

Currently, pciehp_resume() try to hot add device if the slot adapter
status return true. But if there are already some devices exist,
namely list_empty(bus->devices) return false. We should not add the device
again, because the device add action will fail. Also print some uncomfortable
messages like this:
	pciehp 0000:00:1c.1:pcie04: Device 0000:03:00.0 already exists at 0000:03:00, cannot hot-add
	pciehp 0000:00:1c.1:pcie04: Cannot add device at 0000:03:00

Tested-by: Paul Bolle <pebolle@tiscali.nl>
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Cc: Paul Bolle <pebolle@tiscali.nl>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Oliver Neukum <oneukum@suse.de>
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Cc: linux-pci@vger.kernel.org
---
 drivers/pci/hotplug/pciehp_core.c |   10 +++++++---
 1 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c
index 7d72c5e..7fe9dbd 100644
--- a/drivers/pci/hotplug/pciehp_core.c
+++ b/drivers/pci/hotplug/pciehp_core.c
@@ -300,6 +300,7 @@ static int pciehp_resume (struct pcie_device *dev)
 {
 	struct controller *ctrl;
 	struct slot *slot;
+	struct pci_bus *pbus = dev->port->subordinate;
 	u8 status;
 
 	ctrl = get_service_data(dev);
@@ -311,10 +312,13 @@ static int pciehp_resume (struct pcie_device *dev)
 
 	/* Check if slot is occupied */
 	pciehp_get_adapter_status(slot, &status);
-	if (status)
-		pciehp_enable_slot(slot);
-	else
+	if (status) {
+		if (list_empty(&pbus->devices))
+			pciehp_enable_slot(slot);
+	} else if (!list_empty(&pbus->devices)) {
 		pciehp_disable_slot(slot);
+	}
+
 	return 0;
 }
 #endif /* PM */
-- 
1.7.1

[PATCH -v5 2/6] PCI: introduce PCIe Device Serial Number Capability support

[Yijing Wang]

Introduce PCIe Ext Capability Device Serial Number support,
so we can use the unique device serial number to identify
the physical device. During system suspend, if the PCIe
device was removed and reinserted a new same device, during
system resume there is no good way to identify it, maybe
Device Serial Number is a good choice if device support.

Reviewed-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Cc: Paul Bolle <pebolle@tiscali.nl>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Oliver Neukum <oneukum@suse.de>
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Cc: linux-pci@vger.kernel.org
---
 drivers/pci/pci.c   |   29 +++++++++++++++++++++++++++++
 drivers/pci/pci.h   |    2 +-
 drivers/pci/probe.c |    2 ++
 include/linux/pci.h |    1 +
 4 files changed, 33 insertions(+), 1 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 84a3c09..cfc0f7f 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2048,6 +2048,35 @@ void pci_free_cap_save_buffers(struct pci_dev *dev)
 }
 
 /**
+ * pci_device_serial_number - get device serial number
+ * @dev: the PCI device
+ *
+ * return the device serial number if device support,
+ * otherwise return 0.
+ */
+static u64 pci_device_serial_number(struct pci_dev *dev)
+{
+	int pos;
+	u32 lo, hi;
+
+	if (!pci_is_pcie(dev))
+		return 0;
+
+	pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DSN);
+	if (!pos)
+		return 0;
+
+	pci_read_config_dword(dev, pos + 4, &lo);
+	pci_read_config_dword(dev, pos + 8, &hi);
+	return ((u64)hi << 32) | lo;
+}
+
+void pci_dsn_init(struct pci_dev *dev)
+{
+	dev->sn = pci_device_serial_number(dev);
+}
+
+/**
  * pci_configure_ari - enable or disable ARI forwarding
  * @dev: the PCI device
  *
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 816c297..2b2a9cc 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -207,7 +207,7 @@ void __ref __pci_bus_size_bridges(struct pci_bus *bus,
 void __ref __pci_bus_assign_resources(const struct pci_bus *bus,
 				      struct list_head *realloc_head,
 				      struct list_head *fail_head);
-
+void pci_dsn_init(struct pci_dev *dev);
 /**
  * pci_ari_enabled - query ARI forwarding status
  * @bus: the PCI bus
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 83172bc..0c5da76 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1325,6 +1325,8 @@ static void pci_init_capabilities(struct pci_dev *dev)
 	/* Vital Product Data */
 	pci_vpd_pci22_init(dev);
 
+	pci_dsn_init(dev);
+
 	/* Alternative Routing-ID Forwarding */
 	pci_configure_ari(dev);
 
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 0fd1f15..4354eaf 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -342,6 +342,7 @@ struct pci_dev {
 	struct list_head msi_list;
 	struct kset *msi_kset;
 #endif
+	u64 sn;		/* device serial number, 0 if not support */
 	struct pci_vpd *vpd;
 #ifdef CONFIG_PCI_ATS
 	union {
-- 
1.7.1

[PATCH -v5 3/6] PCI: Introduce Vital Product Data Serial Number capability support

[Yijing Wang]

Vital Product Data Serial Number is another capability to support
device serial number, some devices may implement this cap. So we
introduce VPD SN support here to enhance the device change
identification.

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Cc: Paul Bolle <pebolle@tiscali.nl>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Oliver Neukum <oneukum@suse.de>
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Cc: linux-pci@vger.kernel.org
---
 drivers/pci/access.c |    4 ++-
 drivers/pci/pci.h    |    4 +++
 drivers/pci/vpd.c    |   70 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h  |    1 +
 4 files changed, 78 insertions(+), 1 deletions(-)

diff --git a/drivers/pci/access.c b/drivers/pci/access.c
index 1cc2366..0069981 100644
--- a/drivers/pci/access.c
+++ b/drivers/pci/access.c
@@ -194,7 +194,6 @@ PCI_USER_WRITE_CONFIG(dword, u32)
 
 /* VPD access through PCI 2.2+ VPD capability */
 
-#define PCI_VPD_PCI22_SIZE (PCI_VPD_ADDR_MASK + 1)
 
 struct pci_vpd_pci22 {
 	struct pci_vpd base;
@@ -350,6 +349,7 @@ out:
 
 static void pci_vpd_pci22_release(struct pci_dev *dev)
 {
+	kfree(dev->vpd->sn);
 	kfree(container_of(dev->vpd, struct pci_vpd_pci22, base));
 }
 
@@ -377,6 +377,8 @@ int pci_vpd_pci22_init(struct pci_dev *dev)
 	vpd->cap = cap;
 	vpd->busy = false;
 	dev->vpd = &vpd->base;
+
+	pci_vpd_serial_number_init(dev, dev->vpd);
 	return 0;
 }
 
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 2b2a9cc..eb77124 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -83,6 +83,8 @@ static inline bool pci_is_bridge(struct pci_dev *pci_dev)
 	return !!(pci_dev->subordinate);
 }
 
+#define PCI_VPD_PCI22_SIZE (PCI_VPD_ADDR_MASK + 1)
+
 struct pci_vpd_ops {
 	ssize_t (*read)(struct pci_dev *dev, loff_t pos, size_t count, void *buf);
 	ssize_t (*write)(struct pci_dev *dev, loff_t pos, size_t count, const void *buf);
@@ -91,6 +93,7 @@ struct pci_vpd_ops {
 
 struct pci_vpd {
 	unsigned int len;
+	char *sn; /* serial number */
 	const struct pci_vpd_ops *ops;
 	struct bin_attribute *attr; /* descriptor for sysfs VPD entry */
 };
@@ -208,6 +211,7 @@ void __ref __pci_bus_assign_resources(const struct pci_bus *bus,
 				      struct list_head *realloc_head,
 				      struct list_head *fail_head);
 void pci_dsn_init(struct pci_dev *dev);
+struct pci_vpd *pci_vpd_serial_number_init(struct pci_dev *dev, struct pci_vpd *vpd);
 /**
  * pci_ari_enabled - query ARI forwarding status
  * @bus: the PCI bus
diff --git a/drivers/pci/vpd.c b/drivers/pci/vpd.c
index 39b7907..e5a4853 100644
--- a/drivers/pci/vpd.c
+++ b/drivers/pci/vpd.c
@@ -7,6 +7,7 @@
 
 #include <linux/pci.h>
 #include <linux/export.h>
+#include "pci.h"
 
 int pci_vpd_find_tag(const u8 *buf, unsigned int off, unsigned int len, u8 rdt)
 {
@@ -60,3 +61,71 @@ int pci_vpd_find_info_keyword(const u8 *buf, unsigned int off,
 	return -ENOENT;
 }
 EXPORT_SYMBOL_GPL(pci_vpd_find_info_keyword);
+
+/**
+ * pci_vpd_serial_number_init - initialize the device VPD SN
+ * @dev: the PCI device
+ * @vpd: the VPD used to initialize SN
+ *
+ * Initialize the device VPD Serial Number, if vpd passed as NULL,
+ * allocate a new VPD and initialize its SN.
+ */
+struct pci_vpd *pci_vpd_serial_number_init(struct pci_dev *dev,
+				struct pci_vpd *vpd)
+{
+	char *buf = NULL;
+	struct pci_vpd *old = dev->vpd;
+	struct pci_vpd *new = NULL;
+	int cnt, i, end, j, len;
+
+	/* To detect whether the device is changed, we should
+	 * allocate a new VPD to initialize serial number,
+	 * because if the device has been changed, the cap
+	 * info is stale.
+	 */
+	if (!vpd) {
+		dev->vpd = NULL;
+		pci_vpd_pci22_init(dev);
+		if (!dev->vpd)
+			goto fail;
+	}
+
+	new = dev->vpd;
+	buf = kzalloc(PCI_VPD_PCI22_SIZE, GFP_KERNEL);
+	if (!buf)
+		goto fail;
+
+	cnt = pci_read_vpd(dev, 0, PCI_VPD_PCI22_SIZE, buf);
+	if (cnt < 0)
+		goto fail;
+
+	i = pci_vpd_find_tag(buf, 0, PCI_VPD_PCI22_SIZE,
+			PCI_VPD_LRDT_RO_DATA);
+	if (i < 0)
+		goto fail;
+
+	end = i + PCI_VPD_LRDT_TAG_SIZE + pci_vpd_lrdt_size(&buf[i]);
+	i += PCI_VPD_LRDT_TAG_SIZE;
+
+	j = pci_vpd_find_info_keyword(buf, i, end,
+			PCI_VPD_RO_KEYWORD_SN);
+	if (j < 0)
+		goto fail;
+
+	len = pci_vpd_info_field_size(&buf[j]);
+	new->sn = kzalloc(len + 1, GFP_KERNEL);
+	if (!new->sn)
+		goto fail;
+
+	j += PCI_VPD_INFO_FLD_HDR_SIZE;
+	memcpy(new->sn, &buf[j], len);
+	kfree(buf);
+	dev->vpd = old;
+	return new;
+fail:
+	kfree(buf);
+	if (new != old)
+		pci_vpd_release(dev);
+	dev->vpd = old;
+	return NULL;
+}
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 4354eaf..5c38b55 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1781,6 +1781,7 @@ bool pci_acs_path_enabled(struct pci_dev *start,
 
 #define PCI_VPD_RO_KEYWORD_PARTNO	"PN"
 #define PCI_VPD_RO_KEYWORD_MFR_ID	"MN"
+#define PCI_VPD_RO_KEYWORD_SN		"SN"
 #define PCI_VPD_RO_KEYWORD_VENDOR0	"V0"
 #define PCI_VPD_RO_KEYWORD_CHKSUM	"RV"
 
-- 
1.7.1

[PATCH -v5 4/6] PCI: add pci_serial_number_changed() for device change identification

[Yijing Wang]

Sometimes OS do not know the physical device swap,
for instance, some device hotplug during system suspend.
Interrupt can not deliver to OS in some platform.
So we can use pci serial number capability to detect this
issue if device supports serial number.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Cc: Paul Bolle <pebolle@tiscali.nl>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Oliver Neukum <oneukum@suse.de>
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Cc: linux-pci@vger.kernel.org
---
 drivers/pci/access.c |    9 ---------
 drivers/pci/pci.c    |   48 ++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.h    |    8 ++++++++
 include/linux/pci.h  |    2 ++
 4 files changed, 58 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/access.c b/drivers/pci/access.c
index 0069981..7f8df11 100644
--- a/drivers/pci/access.c
+++ b/drivers/pci/access.c
@@ -194,15 +194,6 @@ PCI_USER_WRITE_CONFIG(dword, u32)
 
 /* VPD access through PCI 2.2+ VPD capability */
 
-
-struct pci_vpd_pci22 {
-	struct pci_vpd base;
-	struct mutex lock;
-	u16	flag;
-	bool	busy;
-	u8	cap;
-};
-
 /*
  * Wait for last operation to complete.
  * This code has to spin since there is no other notification from the PCI
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index cfc0f7f..a6ac0c8 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2077,6 +2077,54 @@ void pci_dsn_init(struct pci_dev *dev)
 }
 
 /**
+ * pci_serial_number_changed - check the device SN is changed
+ * @pdev: the PCI device
+ *
+ * check the device serial number is changed.
+ * if device does not support device serial number,
+ * return false.
+ */
+bool pci_serial_number_changed(struct pci_dev *pdev)
+{
+	struct pci_vpd *vpd;
+	u64 old_dsn, new_dsn;
+	int ret;
+
+	/* first check PCIe DSN */
+	old_dsn = pdev->sn;
+	new_dsn = pci_device_serial_number(pdev);
+
+	if (old_dsn != new_dsn)
+		return true;
+	else if (old_dsn)
+		return false;
+
+	/* PCIe DSN does not support, check VPD SN */
+	vpd = pci_vpd_serial_number_init(pdev, NULL);
+	if (!pdev->vpd && !vpd) {
+		/* VPD SN does not support */
+		return false;
+	} else if (pdev->vpd && pdev->vpd->sn && vpd) {
+		ret = strcmp(pdev->vpd->sn, vpd->sn);
+		kfree(vpd->sn);
+		kfree(container_of(vpd, struct pci_vpd_pci22, base));
+		if (!ret)
+			return false;
+		else
+			return true;
+	} else if ((pdev->vpd && pdev->vpd->sn) || (vpd && vpd->sn)) {
+		if (vpd) {
+			kfree(vpd->sn);
+			kfree(container_of(vpd, struct pci_vpd_pci22, base));
+		}
+		return true;
+	}
+
+	return false;
+}
+EXPORT_SYMBOL(pci_serial_number_changed);
+
+/**
  * pci_configure_ari - enable or disable ARI forwarding
  * @dev: the PCI device
  *
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index eb77124..5d1f261 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -98,6 +98,14 @@ struct pci_vpd {
 	struct bin_attribute *attr; /* descriptor for sysfs VPD entry */
 };
 
+struct pci_vpd_pci22 {
+	struct pci_vpd base;
+	struct mutex lock;
+	u16	flag;
+	bool	busy;
+	u8	cap;
+};
+
 int pci_vpd_pci22_init(struct pci_dev *dev);
 static inline void pci_vpd_release(struct pci_dev *dev)
 {
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 5c38b55..9580fa5 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -996,6 +996,8 @@ ssize_t pci_read_vpd(struct pci_dev *dev, loff_t pos, size_t count, void *buf);
 ssize_t pci_write_vpd(struct pci_dev *dev, loff_t pos, size_t count, const void *buf);
 int pci_vpd_truncate(struct pci_dev *dev, size_t size);
 
+bool pci_serial_number_changed(struct pci_dev *pdev);
+
 /* Helper functions for low-level code (drivers/pci/setup-[bus,res].c) */
 resource_size_t pcibios_retrieve_fw_addr(struct pci_dev *dev, int idx);
 void pci_bus_assign_resources(const struct pci_bus *bus);
-- 
1.7.1

[PATCH -v5 5/6] PCI: add inspection of device change in pci_scan_single_device

[Yijing Wang]

Add inspection of device change in pci_scan_single_device.
If device change found, print warning. Because the new device
bind the old device driver, so for safety, we do not remove this
device.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Cc: Paul Bolle <pebolle@tiscali.nl>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Oliver Neukum <oneukum@suse.de>
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Cc: linux-pci@vger.kernel.org
---
 drivers/pci/probe.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 0c5da76..f2812db 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1389,6 +1389,8 @@ struct pci_dev *__ref pci_scan_single_device(struct pci_bus *bus, int devfn)
 
 	dev = pci_get_slot(bus, devfn);
 	if (dev) {
+		if (pci_serial_number_changed(dev))
+			dev_warn(&dev->dev, "device serial number changed!");
 		pci_dev_put(dev);
 		return dev;
 	}
-- 
1.7.1

[PATCH -v5 6/6] PCI,pciehp: identify device change during suspend

[Yijing Wang]

If device was removed from slot and reinsert a new device during
suspend, pciehp can not identify the physical device change now.
So the old driver .resume() method will be called for the new device,
this is bad. If device support device serial number capability,
we can identify this by DSN. So the reasonable way is first remove
the old device, then enable the new device.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Cc: Paul Bolle <pebolle@tiscali.nl>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Oliver Neukum <oneukum@suse.de>
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Cc: linux-pci@vger.kernel.org
---
 drivers/pci/hotplug/pciehp_core.c |   29 ++++++++++++++++++++++++++++-
 1 files changed, 28 insertions(+), 1 deletions(-)

diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c
index 7fe9dbd..7c6e4a6 100644
--- a/drivers/pci/hotplug/pciehp_core.c
+++ b/drivers/pci/hotplug/pciehp_core.c
@@ -301,6 +301,8 @@ static int pciehp_resume (struct pcie_device *dev)
 	struct controller *ctrl;
 	struct slot *slot;
 	struct pci_bus *pbus = dev->port->subordinate;
+	struct pci_dev *pdev;
+	int ret = 0;
 	u8 status;
 
 	ctrl = get_service_data(dev);
@@ -313,8 +315,33 @@ static int pciehp_resume (struct pcie_device *dev)
 	/* Check if slot is occupied */
 	pciehp_get_adapter_status(slot, &status);
 	if (status) {
-		if (list_empty(&pbus->devices))
+		if (list_empty(&pbus->devices)) {
 			pciehp_enable_slot(slot);
+			return 0;
+		}
+
+		pdev = pci_get_slot(pbus, PCI_DEVFN(0, 0));
+		if (!pdev)
+			return 0;
+		ret = pci_serial_number_changed(pdev);
+		pci_dev_put(pdev);
+		if (ret) {
+			/*
+			 * first power off slot, avoid the old driver
+			 * .remove() method touch the new hardware
+			 */
+			if (POWER_CTRL(ctrl)) {
+				ret = pciehp_power_off_slot(slot);
+				if (ret) {
+					ctrl_err(ctrl,
+						"Issue of Slot Disable command failed\n");
+					return 0;
+				}
+				msleep(1000);
+				pciehp_unconfigure_device(slot);
+				pciehp_enable_slot(slot);
+			}
+		}
 	} else if (!list_empty(&pbus->devices)) {
 		pciehp_disable_slot(slot);
 	}
-- 
1.7.1

Re: [PATCH -v5 0/6] Use PCI Serial Number to identify device change

[Paul Bolle]

On Mon, 2013-08-05 at 16:54 +0800, Yijing Wang wrote:
> v4->v5: update patch 6/6, correct some mistakes in this patch, thank Paul for help
> 		test this series.
>[...]
> I tested to apply this series to Linus 3.11-rc4 and Bjorn's pci-next branch, and
> result is ok.

And I tested this series on top of v3.10.5 (on x86_64). It applied
cleanly, and it compiled cleanly. No pciehp related errors at resume.
And, more importantly, the severe issues of v4 are gone: a few suspend
and resume cycles showed no obvious problems.

Thanks!


Paul Bolle

Re: [PATCH -v5 0/6] Use PCI Serial Number to identify device change

[Yijing Wang]

On 2013/8/5 20:01, Paul Bolle wrote:
> On Mon, 2013-08-05 at 16:54 +0800, Yijing Wang wrote:
>> v4->v5: update patch 6/6, correct some mistakes in this patch, thank Paul for help
>> 		test this series.
>> [...]
>> I tested to apply this series to Linus 3.11-rc4 and Bjorn's pci-next branch, and
>> result is ok.
> 
> And I tested this series on top of v3.10.5 (on x86_64). It applied
> cleanly, and it compiled cleanly. No pciehp related errors at resume.
> And, more importantly, the severe issues of v4 are gone: a few suspend
> and resume cycles showed no obvious problems.
> 

Hi Paul,
   Thanks for your test very much!

Thanks!
Yijing.

> 


-- 
Thanks!
Yijing