[PATCH] clockevent: don't remove broadcast device when cpu is dead

[PATCH] clockevent: don't remove broadcast device when cpu is dead

[Xiaotian Feng]

Marc reported BUG during shutdown, after debugging, kernel is trying
to remove a broadcast device which mode is CLOCK_EVT_MODE_ONESHOT.

The root cause for this bug is that in clockevents_notify,
"cpumask_weight(dev->cpumask) == 1" is always true even if dev is a
broadcast device. We need to use tick_is_broadcast_device to check
if it is a broadcast device.

Reported-by: Marc Dionne <marc.c.dionne@gmail.com>
Tested-by: Marc Dionne <marc.c.dionne@gmail.com>
Signed-off-by: Xiaotian Feng <dfeng@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Magnus Damm <damm@igel.co.jp>
Cc: H Hartley Sweeten <hsweeten@visionengravers.com>
---
 kernel/time/clockevents.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
index 6f740d9..0223d83 100644
--- a/kernel/time/clockevents.c
+++ b/kernel/time/clockevents.c
@@ -259,7 +259,7 @@ void clockevents_notify(unsigned long reason, void *arg)
 		cpu = *((int *)arg);
 		list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) {
 			if (cpumask_test_cpu(cpu, dev->cpumask) &&
-			    cpumask_weight(dev->cpumask) == 1) {
+			    !tick_is_broadcast_device(dev)) {
 				BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
 				list_del(&dev->list);
 			}
-- 
1.6.5.2

Re: [PATCH] clockevent: don't remove broadcast device when cpu is dead

[Xiaotian Feng]

Hi:

     Any comments on the patch? Marc confirmed this patch also fixed his 
hang at suspend/resume stage. Thanks.

     (Cc'ed stable@kernel.org)

Regards
Xiaotian

On 01/07/2010 11:22 AM, Xiaotian Feng wrote:
> Marc reported BUG during shutdown, after debugging, kernel is trying
> to remove a broadcast device which mode is CLOCK_EVT_MODE_ONESHOT.
>
> The root cause for this bug is that in clockevents_notify,
> "cpumask_weight(dev->cpumask) == 1" is always true even if dev is a
> broadcast device. We need to use tick_is_broadcast_device to check
> if it is a broadcast device.
>
> Reported-by: Marc Dionne<marc.c.dionne@gmail.com>
> Tested-by: Marc Dionne<marc.c.dionne@gmail.com>
> Signed-off-by: Xiaotian Feng<dfeng@redhat.com>
> Cc: Thomas Gleixner<tglx@linutronix.de>
> Cc: Magnus Damm<damm@igel.co.jp>
> Cc: H Hartley Sweeten<hsweeten@visionengravers.com>
> ---
>   kernel/time/clockevents.c |    2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
> index 6f740d9..0223d83 100644
> --- a/kernel/time/clockevents.c
> +++ b/kernel/time/clockevents.c
> @@ -259,7 +259,7 @@ void clockevents_notify(unsigned long reason, void *arg)
>   		cpu = *((int *)arg);
>   		list_for_each_entry_safe(dev, tmp,&clockevent_devices, list) {
>   			if (cpumask_test_cpu(cpu, dev->cpumask)&&
> -			    cpumask_weight(dev->cpumask) == 1) {
> +			    !tick_is_broadcast_device(dev)) {
>   				BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
>   				list_del(&dev->list);
>   			}
>    

Re: [PATCH] clockevent: don't remove broadcast device when cpu is dead

[Thomas Gleixner]

On Thu, 7 Jan 2010, Xiaotian Feng wrote:

> Marc reported BUG during shutdown, after debugging, kernel is trying
> to remove a broadcast device which mode is CLOCK_EVT_MODE_ONESHOT.
> 
> The root cause for this bug is that in clockevents_notify,
> "cpumask_weight(dev->cpumask) == 1" is always true even if dev is a

Why is cpumask_weight(dev->cpumask) == 1 always true when we shutdown
a non boot cpu ?

The broadcast device is not a per cpu device and the cpumask should
not only contain the CPU which is shut down !

The patch is papering over the real problem.

Marc, can you please apply the following debug patch and provide the
dmesg outputs from boot and shutdown ?

Thanks,

	tglx
---

Index: linux-2.6-tip/kernel/time/clockevents.c
===================================================================
--- linux-2.6-tip.orig/kernel/time/clockevents.c
+++ linux-2.6-tip/kernel/time/clockevents.c
@@ -186,7 +186,7 @@ void clockevents_register_device(struct 
 	BUG_ON(!dev->cpumask);
 
 	raw_spin_lock_irqsave(&clockevents_lock, flags);
-
+	printk(KERN_ERR "CE register %p %s\n", dev, dev->name);
 	list_add(&dev->list, &clockevent_devices);
 	clockevents_do_notify(CLOCK_EVT_NOTIFY_ADD, dev);
 	clockevents_notify_released();
@@ -220,6 +220,7 @@ void clockevents_exchange_device(struct 
 	 * released list and do a notify add later.
 	 */
 	if (old) {
+		printk(KERN_INFO "CE Release %p %s\n", old, old->name);
 		clockevents_set_mode(old, CLOCK_EVT_MODE_UNUSED);
 		list_del(&old->list);
 		list_add(&old->list, &clockevents_released);
@@ -260,6 +261,13 @@ void clockevents_notify(unsigned long re
 		list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) {
 			if (cpumask_test_cpu(cpu, dev->cpumask) &&
 			    cpumask_weight(dev->cpumask) == 1) {
+				if (dev->mode != CLOCK_EVT_MODE_UNUSED) {
+					printk(KERN_INFO
+					       "CE Remove %p %s bc: %d\n",
+					       dev, dev->name,
+					       tick_is_broadcast_device(dev));
+					continue;
+				}
 				BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
 				list_del(&dev->list);
 			}
Index: linux-2.6-tip/kernel/time/tick-broadcast.c
===================================================================
--- linux-2.6-tip.orig/kernel/time/tick-broadcast.c
+++ linux-2.6-tip/kernel/time/tick-broadcast.c
@@ -72,6 +72,8 @@ int tick_check_broadcast_device(struct c
 	     (dev->features & CLOCK_EVT_FEAT_C3STOP))
 		return 0;
 
+	printk(KERN_INFO "CE set broadcast %p %s\n", dev, dev->name);
+
 	clockevents_exchange_device(NULL, dev);
 	tick_broadcast_device.evtdev = dev;
 	if (!cpumask_empty(tick_get_broadcast_mask()))

Re: [PATCH] clockevent: don't remove broadcast device when cpu is dead

[Marc Dionne]

[-- Attachment #1: Type: text/plain, Size: 1042 bytes --]

On Tue, Jan 12, 2010 at 8:20 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> On Thu, 7 Jan 2010, Xiaotian Feng wrote:
>
>> Marc reported BUG during shutdown, after debugging, kernel is trying
>> to remove a broadcast device which mode is CLOCK_EVT_MODE_ONESHOT.
>>
>> The root cause for this bug is that in clockevents_notify,
>> "cpumask_weight(dev->cpumask) == 1" is always true even if dev is a
>
> Why is cpumask_weight(dev->cpumask) == 1 always true when we shutdown
> a non boot cpu ?
>
> The broadcast device is not a per cpu device and the cpumask should
> not only contain the CPU which is shut down !
>
> The patch is papering over the real problem.
>
> Marc, can you please apply the following debug patch and provide the
> dmesg outputs from boot and shutdown ?
>
> Thanks,
>
>        tglx

Full dmesg output from boot is attached.

On shutdown I wrote down the message just before the BUG by hand
(after modifying your patch a little to let it BUG):

CE Remove ffffffff81a44100 hpet bc: 1

Marc

[-- Attachment #2: boot_ce --]
[-- Type: application/octet-stream, Size: 44188 bytes --]

Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.33-rc3-marco (root@tornado) (gcc version 4.4.2 20091222 (Red Hat 4.4.2-20) (GCC) ) #23 SMP Tue Jan 12 18:21:04 EST 2010
Command line: ro root=UUID=2013c140-3e2e-4b04-aaa0-86893fba78df vga=0x31b selinux=1 rhgb quiet SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=us 3
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000bf780000 (usable)
 BIOS-e820: 00000000bf780000 - 00000000bf78e000 (ACPI data)
 BIOS-e820: 00000000bf78e000 - 00000000bf7d0000 (ACPI NVS)
 BIOS-e820: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
 BIOS-e820: 00000000bf7ec000 - 00000000c0000000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ffc00000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 00000001c0000000 (usable)
NX (Execute Disable) protection: active
DMI present.
AMI BIOS detected: BIOS may corrupt low RAM, working around it.
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
No AGP bridge found
last_pfn = 0x1c0000 max_arch_pfn = 0x400000000
MTRR default type: uncachable
MTRR fixed ranges enabled:
  00000-9FFFF write-back
  A0000-BFFFF uncachable
  C0000-CFFFF write-protect
  D0000-DFFFF uncachable
  E0000-E3FFF write-protect
  E4000-EFFFF write-through
  F0000-FFFFF write-protect
MTRR variable ranges enabled:
  0 base 1C0000000 mask FC0000000 uncachable
  1 base 000000000 mask E00000000 write-back
  2 base 0C0000000 mask FC0000000 uncachable
  3 base 0BF800000 mask FFF800000 uncachable
  4 disabled
  5 disabled
  6 disabled
  7 disabled
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
original variable MTRRs
reg 0, base: 7GB, range: 1GB, type UC
reg 1, base: 0GB, range: 8GB, type WB
reg 2, base: 3GB, range: 1GB, type UC
reg 3, base: 3064MB, range: 8MB, type UC
total RAM covered: 6136M
Found optimal setting for mtrr clean up
 gran_size: 64K 	chunk_size: 16M 	num_reg: 5  	lose cover RAM: 0G
New variable MTRRs
reg 0, base: 0GB, range: 2GB, type WB
reg 1, base: 2GB, range: 1GB, type WB
reg 2, base: 3064MB, range: 8MB, type UC
reg 3, base: 4GB, range: 2GB, type WB
reg 4, base: 6GB, range: 1GB, type WB
e820 update range: 00000000bf800000 - 0000000100000000 (usable) ==> (reserved)
last_pfn = 0xbf780 max_arch_pfn = 0x400000000
initial memory mapped : 0 - 20000000
found SMP MP-table at [ffff8800000ff780] ff780
init_memory_mapping: 0000000000000000-00000000bf780000
 0000000000 - 00bf600000 page 2M
 00bf600000 - 00bf780000 page 4k
kernel direct mapping tables up to bf780000 @ 16000-1b000
init_memory_mapping: 0000000100000000-00000001c0000000
 0100000000 - 01c0000000 page 2M
kernel direct mapping tables up to 1c0000000 @ 19000-21000
RAMDISK: 371fa000 - 37fef5d0
ACPI: RSDP 00000000000fa040 00014 (v00 ACPIAM)
ACPI: RSDT 00000000bf780000 00038 (v01 7593MS A7593100 20090417 MSFT 00000097)
ACPI: FACP 00000000bf780200 00084 (v01 7593MS A7593100 20090417 MSFT 00000097)
ACPI: DSDT 00000000bf780460 06D02 (v01  A7593 A7593100 00000100 INTL 20051117)
ACPI: FACS 00000000bf78e000 00040
ACPI: APIC 00000000bf780390 0008C (v01 7593MS A7593100 20090417 MSFT 00000097)
ACPI: MCFG 00000000bf780420 0003C (v01 7593MS OEMMCFG  20090417 MSFT 00000097)
ACPI: OEMB 00000000bf78e040 0007A (v01 7593MS A7593100 20090417 MSFT 00000097)
ACPI: SSDT 00000000bf790140 01298 (v01 DpgPmm    CpuPm 00000012 INTL 20051117)
ACPI: Local APIC address 0xfee00000
No NUMA configuration found
Faking a node at 0000000000000000-00000001c0000000
Bootmem setup node 0 0000000000000000-00000001c0000000
  NODE_DATA [000000000001c000 - 0000000000030fff]
  bootmap [0000000000031000 -  0000000000068fff] pages 38
(13 early reservations) ==> bootmem [0000000000 - 01c0000000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0001000000 - 0001dccdc8]    TEXT DATA BSS ==> [0001000000 - 0001dccdc8]
  #2 [00371fa000 - 0037fef5d0]          RAMDISK ==> [00371fa000 - 0037fef5d0]
  #3 [0001dcd000 - 0001dcd127]              BRK ==> [0001dcd000 - 0001dcd127]
  #4 [00000ff790 - 0000100000]    BIOS reserved ==> [00000ff790 - 0000100000]
  #5 [00000ff780 - 00000ff790]     MP-table mpf ==> [00000ff780 - 00000ff790]
  #6 [000009fc00 - 00000fddb0]    BIOS reserved ==> [000009fc00 - 00000fddb0]
  #7 [00000fdf5c - 00000ff780]    BIOS reserved ==> [00000fdf5c - 00000ff780]
  #8 [00000fddb0 - 00000fdf5c]     MP-table mpc ==> [00000fddb0 - 00000fdf5c]
  #9 [0000010000 - 0000012000]       TRAMPOLINE ==> [0000010000 - 0000012000]
  #10 [0000012000 - 0000016000]      ACPI WAKEUP ==> [0000012000 - 0000016000]
  #11 [0000016000 - 0000019000]          PGTABLE ==> [0000016000 - 0000019000]
  #12 [0000019000 - 000001c000]          PGTABLE ==> [0000019000 - 000001c000]
 [ffffea0000000000-ffffea00061fffff] PMD -> [ffff880028600000-ffff88002d9fffff] on node 0
Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x001c0000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
    0: 0x00000010 -> 0x0000009f
    0: 0x00000100 -> 0x000bf780
    0: 0x00100000 -> 0x001c0000
On node 0 totalpages: 1570575
  DMA zone: 56 pages used for memmap
  DMA zone: 108 pages reserved
  DMA zone: 3819 pages, LIFO batch:0
  DMA32 zone: 14280 pages used for memmap
  DMA32 zone: 765880 pages, LIFO batch:31
  Normal zone: 10752 pages used for memmap
  Normal zone: 775680 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 8 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 24
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
PM: Registered nosave memory: 00000000bf780000 - 00000000bf78e000
PM: Registered nosave memory: 00000000bf78e000 - 00000000bf7d0000
PM: Registered nosave memory: 00000000bf7d0000 - 00000000bf7e0000
PM: Registered nosave memory: 00000000bf7e0000 - 00000000bf7ec000
PM: Registered nosave memory: 00000000bf7ec000 - 00000000c0000000
PM: Registered nosave memory: 00000000c0000000 - 00000000fee00000
PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
PM: Registered nosave memory: 00000000fee01000 - 00000000ffc00000
PM: Registered nosave memory: 00000000ffc00000 - 0000000100000000
Allocating PCI resources starting at c0000000 (gap: c0000000:3ee00000)
Booting paravirtualized kernel on bare hardware
setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:8 nr_node_ids:1
PERCPU: Embedded 30 pages/cpu @ffff880028200000 s90584 r8192 d24104 u262144
pcpu-alloc: s90584 r8192 d24104 u262144 alloc=1*2097152
pcpu-alloc: [0] 0 1 2 3 4 5 6 7 
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1545379
Policy zone: Normal
Kernel command line: ro root=UUID=2013c140-3e2e-4b04-aaa0-86893fba78df vga=0x31b selinux=1 rhgb quiet SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=us 3
PID hash table entries: 4096 (order: 3, 32768 bytes)
Checking aperture...
No AGP bridge found
Calgary: detecting Calgary via BIOS EBDA area
Calgary: Unable to locate Rio Grande table in EBDA - bailing!
Memory: 6100544k/7340032k available (4921k kernel code, 1057732k absent, 181756k reserved, 6367k data, 1564k init)
SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
Hierarchical RCU implementation.
NR_IRQS:4352 nr_irqs:472
Console: colour dummy device 80x25
console [tty0] enabled
allocated 62914560 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
CE register ffffffff81a41530 pit
Fast TSC calibration using PIT
Detected 2673.529 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 5347.05 BogoMIPS (lpj=2673529)
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 9 MCE banks
CPU0: Thermal monitoring enabled (TM1)
CPU 0 MCA banks CMCI:2 CMCI:3 CMCI:5 CMCI:6 SHD:8
using mwait in idle threads.
Performance Events: Nehalem/Corei7 events, Intel PMU driver.
... version:                3
... bit width:              48
... generic registers:      4
... value mask:             0000ffffffffffff
... max period:             000000007fffffff
... fixed-purpose events:   3
... event mask:             000000070000000f
ACPI: Core revision 20091214
ftrace: converting mcount calls to 0f 1f 44 00 00
ftrace: allocating 25542 entries in 101 pages
Setting APIC routing to flat
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
CE register ffff88002820e470 lapic
CE Release ffffffff81a41530 pit
CE set broadcast ffffffff81a41530 pit
Booting Node   0, Processors  #1
CPU 1 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
CE register ffff88002824e470 lapic
 #2
CPU 2 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
CE register ffff88002828e470 lapic
 #3
CPU 3 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
CE register ffff8800282ce470 lapic
 #4
CPU 4 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
CE register ffff88002830e470 lapic
 #5
CPU 5 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
CE register ffff88002834e470 lapic
 #6
CPU 6 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
CE register ffff88002838e470 lapic
 #7 Ok.
CPU 7 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
CE register ffff8800283ce470 lapic
Brought up 8 CPUs
Total of 8 processors activated (42759.78 BogoMIPS).
Time: 23:25:01  Date: 01/12/10
NET: Registered protocol family 16
ACPI: bus type pci registered
dca service started, version 1.12.1
PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
PCI: not using MMCONFIG
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Executed 1 blocks of module-level executable AML code
ACPI Warning: Incorrect checksum in table [OEMB] - B8, should be B7 (20091214/tbutils-314)
ACPI: SSDT 00000000bf78e0c0 00403 (v01 DpgPmm  P001Ist 00000011 INTL 20051117)
ACPI: SSDT 00000000bf78e4d0 00403 (v01 DpgPmm  P002Ist 00000012 INTL 20051117)
ACPI: SSDT 00000000bf78e8e0 00403 (v01 DpgPmm  P003Ist 00000012 INTL 20051117)
ACPI: SSDT 00000000bf78ecf0 00403 (v01 DpgPmm  P004Ist 00000012 INTL 20051117)
ACPI: SSDT 00000000bf78f100 00403 (v01 DpgPmm  P005Ist 00000012 INTL 20051117)
ACPI: SSDT 00000000bf78f510 00403 (v01 DpgPmm  P006Ist 00000012 INTL 20051117)
ACPI: SSDT 00000000bf78f920 00403 (v01 DpgPmm  P007Ist 00000012 INTL 20051117)
ACPI: SSDT 00000000bf78fd30 00403 (v01 DpgPmm  P008Ist 00000012 INTL 20051117)
ACPI: Interpreter enabled
ACPI: (supports S0 S3 S4 S5)
ACPI: Using IOAPIC for interrupt routing
PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI motherboard resources
ACPI: No dock devices found.
ACPI: PCI Root Bridge [PCI0] (0000:00)
pci_root PNP0A08:00: ignoring host bridge windows from ACPI; boot with "pci=use_crs" to use them
pci_root PNP0A08:00: host bridge window [io  0x0000-0x0cf7] (ignored)
pci_root PNP0A08:00: host bridge window [io  0x0d00-0xffff] (ignored)
pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff] (ignored)
pci_root PNP0A08:00: host bridge window [mem 0x000d0000-0x000dffff] (ignored)
pci_root PNP0A08:00: host bridge window [mem 0xc0000000-0xdfffffff] (ignored)
pci_root PNP0A08:00: host bridge window [mem 0xf0000000-0xfed8ffff] (ignored)
pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
pci 0000:00:00.0: PME# disabled
pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
pci 0000:00:01.0: PME# disabled
pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
pci 0000:00:03.0: PME# disabled
pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
pci 0000:00:07.0: PME# disabled
IOH bus: [00, fb]
IOH bus: 00 index 0 io port: [0, efff]
IOH bus: 00 index 1 mmio: [d0000000, fbffffff]
pci 0000:00:1a.0: reg 20: [io  0xac00-0xac1f]
pci 0000:00:1a.1: reg 20: [io  0xa880-0xa89f]
pci 0000:00:1a.2: reg 20: [io  0xa800-0xa81f]
pci 0000:00:1a.7: reg 10: [mem 0xf8ffe000-0xf8ffe3ff]
pci 0000:00:1a.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1a.7: PME# disabled
pci 0000:00:1b.0: reg 10: [mem 0xf8ff8000-0xf8ffbfff 64bit]
pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1b.0: PME# disabled
pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.0: PME# disabled
pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.1: PME# disabled
pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.4: PME# disabled
pci 0000:00:1c.5: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.5: PME# disabled
pci 0000:00:1d.0: reg 20: [io  0xa480-0xa49f]
pci 0000:00:1d.1: reg 20: [io  0xa400-0xa41f]
pci 0000:00:1d.2: reg 20: [io  0xa080-0xa09f]
pci 0000:00:1d.7: reg 10: [mem 0xf8ffc000-0xf8ffc3ff]
pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1d.7: PME# disabled
pci 0000:00:1f.0: Force enabled HPET at 0xfed00000
pci 0000:00:1f.0: quirk: [io  0x0800-0x087f] claimed by ICH6 ACPI/GPIO/TCO
pci 0000:00:1f.0: quirk: [io  0x0500-0x053f] claimed by ICH6 GPIO
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 1 PIO at 0a00 (mask 00ff)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 2 PIO at 0a00 (mask 0017)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 3 PIO at 4700 (mask 00ff)
pci 0000:00:1f.2: reg 10: [io  0xa000-0xa007]
pci 0000:00:1f.2: reg 14: [io  0x9c00-0x9c03]
pci 0000:00:1f.2: reg 18: [io  0x9880-0x9887]
pci 0000:00:1f.2: reg 1c: [io  0x9800-0x9803]
pci 0000:00:1f.2: reg 20: [io  0x9480-0x948f]
pci 0000:00:1f.2: reg 24: [io  0x9400-0x940f]
pci 0000:00:1f.3: reg 10: [mem 0xf8ff7c00-0xf8ff7cff 64bit]
pci 0000:00:1f.3: reg 20: [io  0x0400-0x041f]
pci 0000:00:1f.5: reg 10: [io  0x9000-0x9007]
pci 0000:00:1f.5: reg 14: [io  0x8c00-0x8c03]
pci 0000:00:1f.5: reg 18: [io  0x8880-0x8887]
pci 0000:00:1f.5: reg 1c: [io  0x8800-0x8803]
pci 0000:00:1f.5: reg 20: [io  0x8480-0x848f]
pci 0000:00:1f.5: reg 24: [io  0x8400-0x840f]
PCI: peer root bus 00 res updated from pci conf
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:02:00.0: reg 10: [mem 0xfa000000-0xfaffffff]
pci 0000:02:00.0: reg 14: [mem 0xd0000000-0xdfffffff 64bit pref]
pci 0000:02:00.0: reg 1c: [mem 0xf9000000-0xf9ffffff 64bit]
pci 0000:02:00.0: reg 24: [io  0xbc00-0xbc7f]
pci 0000:02:00.0: reg 30: [mem 0xfbbe0000-0xfbbfffff pref]
pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
pci 0000:00:03.0: PCI bridge to [bus 02-02]
pci 0000:00:03.0:   bridge window [io  0xb000-0xbfff]
pci 0000:00:03.0:   bridge window [mem 0xf9000000-0xfbbfffff]
pci 0000:00:03.0:   bridge window [mem 0xd0000000-0xdfffffff 64bit pref]
pci 0000:00:07.0: PCI bridge to [bus 03-03]
pci 0000:00:1c.0: PCI bridge to [bus 04-04]
pci 0000:05:00.0: reg 24: [mem 0xfbcfe000-0xfbcfffff]
pci 0000:05:00.0: PME# supported from D3hot
pci 0000:05:00.0: PME# disabled
pci 0000:05:00.1: reg 10: [io  0xcc00-0xcc07]
pci 0000:05:00.1: reg 14: [io  0xc880-0xc883]
pci 0000:05:00.1: reg 18: [io  0xc800-0xc807]
pci 0000:05:00.1: reg 1c: [io  0xc480-0xc483]
pci 0000:05:00.1: reg 20: [io  0xc400-0xc40f]
pci 0000:05:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
pci 0000:00:1c.1: PCI bridge to [bus 05-05]
pci 0000:00:1c.1:   bridge window [io  0xc000-0xcfff]
pci 0000:00:1c.1:   bridge window [mem 0xfbc00000-0xfbcfffff]
pci 0000:06:00.0: reg 10: [io  0xd800-0xd8ff]
pci 0000:06:00.0: reg 18: [mem 0xfbdff000-0xfbdfffff 64bit]
pci 0000:06:00.0: reg 20: [mem 0xf7ff0000-0xf7ffffff 64bit pref]
pci 0000:06:00.0: reg 30: [mem 0xfbdc0000-0xfbddffff pref]
pci 0000:06:00.0: supports D1 D2
pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold
pci 0000:06:00.0: PME# disabled
pci 0000:00:1c.4: PCI bridge to [bus 06-06]
pci 0000:00:1c.4:   bridge window [io  0xd000-0xdfff]
pci 0000:00:1c.4:   bridge window [mem 0xfbd00000-0xfbdfffff]
pci 0000:00:1c.4:   bridge window [mem 0xf7f00000-0xf7ffffff 64bit pref]
pci 0000:07:00.0: reg 10: [mem 0xfbeff800-0xfbefffff 64bit]
pci 0000:07:00.0: reg 18: [io  0xe800-0xe8ff]
pci 0000:07:00.0: supports D2
pci 0000:07:00.0: PME# supported from D2 D3hot D3cold
pci 0000:07:00.0: PME# disabled
pci 0000:00:1c.5: PCI bridge to [bus 07-07]
pci 0000:00:1c.5:   bridge window [io  0xe000-0xefff]
pci 0000:00:1c.5:   bridge window [mem 0xfbe00000-0xfbefffff]
pci 0000:00:1e.0: PCI bridge to [bus 08-08] (subtractive decode)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P4._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P5._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P8._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P9._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE3._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE7._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs *5)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 11 12 *14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKG] (IRQs *3 4 6 7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 6 *7 10 11 12 14 15)
vgaarb: device added: PCI:0000:02:00.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: pci_cache_line_size set to 64 bytes
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
CE register ffffffff81a44100 hpet
CE set broadcast ffffffff81a44100 hpet
hpet clockevent registered
HPET: 4 timers in total, 0 timers will be used for per-cpu timer
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
hpet0: 4 comparators, 64-bit 14.318180 MHz counter
Switching to clocksource tsc
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 13 devices
ACPI: ACPI bus type pnp unregistered
system 00:01: [mem 0xfbf00000-0xfbffffff] has been reserved
system 00:01: [mem 0xfc000000-0xfcffffff] has been reserved
system 00:01: [mem 0xfd000000-0xfdffffff] has been reserved
system 00:01: [mem 0xfe000000-0xfebfffff] has been reserved
system 00:01: [mem 0xfec8a000-0xfec8afff] has been reserved
system 00:01: [mem 0xfed10000-0xfed10fff] has been reserved
system 00:08: [io  0x0a00-0x0adf] has been reserved
system 00:08: [io  0x0ae0-0x0aef] has been reserved
system 00:09: [io  0x04d0-0x04d1] has been reserved
system 00:09: [io  0x0800-0x087f] has been reserved
system 00:09: [io  0x0500-0x057f] could not be reserved
system 00:09: [mem 0xfed1c000-0xfed1ffff] has been reserved
system 00:09: [mem 0xfed10000-0xfed103ff] has been reserved
system 00:09: [mem 0xfed10400-0xfed107ff] has been reserved
system 00:09: [mem 0xfed10800-0xfed10bff] has been reserved
system 00:09: [mem 0xfed10c00-0xfed10fff] has been reserved
system 00:09: [mem 0xfec8a000-0xfec8a3ff] has been reserved
system 00:09: [mem 0xfec8a400-0xfec8a7ff] has been reserved
system 00:09: [mem 0xfec8a800-0xfec8abff] has been reserved
system 00:09: [mem 0xfec8ac00-0xfec8afff] has been reserved
system 00:09: [mem 0xfed20000-0xfed3ffff] has been reserved
system 00:09: [mem 0xfed45000-0xfed89fff] has been reserved
system 00:09: [mem 0xfed20000-0xfed3ffff] has been reserved
system 00:09: [mem 0xfed40000-0xfed8ffff] could not be reserved
system 00:0a: [mem 0xfec00000-0xfec00fff] could not be reserved
system 00:0a: [mem 0xfee00000-0xfee00fff] has been reserved
system 00:0b: [mem 0xe0000000-0xefffffff] has been reserved
system 00:0c: [mem 0x00000000-0x0009ffff] could not be reserved
system 00:0c: [mem 0x000c0000-0x000cffff] has been reserved
system 00:0c: [mem 0x000e0000-0x000fffff] could not be reserved
system 00:0c: [mem 0x00100000-0xbfffffff] could not be reserved
system 00:0c: [mem 0xfed90000-0xffffffff] could not be reserved
pci 0000:00:1c.0: BAR 8: assigned [mem 0xf0000000-0xf01fffff]
pci 0000:00:1c.0: BAR 9: assigned [mem 0xf0200000-0xf03fffff 64bit pref]
pci 0000:00:1c.1: BAR 9: assigned [mem 0xf0400000-0xf05fffff 64bit pref]
pci 0000:00:1c.5: BAR 9: assigned [mem 0xf0600000-0xf07fffff 64bit pref]
pci 0000:00:1c.0: BAR 7: assigned [io  0x1000-0x1fff]
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0:   bridge window [io  disabled]
pci 0000:00:01.0:   bridge window [mem disabled]
pci 0000:00:01.0:   bridge window [mem pref disabled]
pci 0000:00:03.0: PCI bridge to [bus 02-02]
pci 0000:00:03.0:   bridge window [io  0xb000-0xbfff]
pci 0000:00:03.0:   bridge window [mem 0xf9000000-0xfbbfffff]
pci 0000:00:03.0:   bridge window [mem 0xd0000000-0xdfffffff 64bit pref]
pci 0000:00:07.0: PCI bridge to [bus 03-03]
pci 0000:00:07.0:   bridge window [io  disabled]
pci 0000:00:07.0:   bridge window [mem disabled]
pci 0000:00:07.0:   bridge window [mem pref disabled]
pci 0000:00:1c.0: PCI bridge to [bus 04-04]
pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
pci 0000:00:1c.0:   bridge window [mem 0xf0000000-0xf01fffff]
pci 0000:00:1c.0:   bridge window [mem 0xf0200000-0xf03fffff 64bit pref]
pci 0000:00:1c.1: PCI bridge to [bus 05-05]
pci 0000:00:1c.1:   bridge window [io  0xc000-0xcfff]
pci 0000:00:1c.1:   bridge window [mem 0xfbc00000-0xfbcfffff]
pci 0000:00:1c.1:   bridge window [mem 0xf0400000-0xf05fffff 64bit pref]
pci 0000:00:1c.4: PCI bridge to [bus 06-06]
pci 0000:00:1c.4:   bridge window [io  0xd000-0xdfff]
pci 0000:00:1c.4:   bridge window [mem 0xfbd00000-0xfbdfffff]
pci 0000:00:1c.4:   bridge window [mem 0xf7f00000-0xf7ffffff 64bit pref]
pci 0000:00:1c.5: PCI bridge to [bus 07-07]
pci 0000:00:1c.5:   bridge window [io  0xe000-0xefff]
pci 0000:00:1c.5:   bridge window [mem 0xfbe00000-0xfbefffff]
pci 0000:00:1c.5:   bridge window [mem 0xf0600000-0xf07fffff 64bit pref]
pci 0000:00:1e.0: PCI bridge to [bus 08-08]
pci 0000:00:1e.0:   bridge window [io  disabled]
pci 0000:00:1e.0:   bridge window [mem disabled]
pci 0000:00:1e.0:   bridge window [mem pref disabled]
pci 0000:00:01.0: setting latency timer to 64
pci 0000:00:03.0: setting latency timer to 64
pci 0000:00:07.0: setting latency timer to 64
pci 0000:00:1c.0: enabling device (0104 -> 0107)
  alloc irq_desc for 17 on node -1
  alloc kstat_irqs on node -1
pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.0: setting latency timer to 64
  alloc irq_desc for 16 on node -1
  alloc kstat_irqs on node -1
pci 0000:00:1c.1: PCI INT B -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:1c.1: setting latency timer to 64
pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.4: setting latency timer to 64
pci 0000:00:1c.5: PCI INT B -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:1c.5: setting latency timer to 64
pci 0000:00:1e.0: setting latency timer to 64
pci_bus 0000:00: resource 0 [io  0x0000-0xefff]
pci_bus 0000:00: resource 1 [mem 0xd0000000-0xfbffffff]
pci_bus 0000:02: resource 0 [io  0xb000-0xbfff]
pci_bus 0000:02: resource 1 [mem 0xf9000000-0xfbbfffff]
pci_bus 0000:02: resource 2 [mem 0xd0000000-0xdfffffff 64bit pref]
pci_bus 0000:04: resource 0 [io  0x1000-0x1fff]
pci_bus 0000:04: resource 1 [mem 0xf0000000-0xf01fffff]
pci_bus 0000:04: resource 2 [mem 0xf0200000-0xf03fffff 64bit pref]
pci_bus 0000:05: resource 0 [io  0xc000-0xcfff]
pci_bus 0000:05: resource 1 [mem 0xfbc00000-0xfbcfffff]
pci_bus 0000:05: resource 2 [mem 0xf0400000-0xf05fffff 64bit pref]
pci_bus 0000:06: resource 0 [io  0xd000-0xdfff]
pci_bus 0000:06: resource 1 [mem 0xfbd00000-0xfbdfffff]
pci_bus 0000:06: resource 2 [mem 0xf7f00000-0xf7ffffff 64bit pref]
pci_bus 0000:07: resource 0 [io  0xe000-0xefff]
pci_bus 0000:07: resource 1 [mem 0xfbe00000-0xfbefffff]
pci_bus 0000:07: resource 2 [mem 0xf0600000-0xf07fffff 64bit pref]
pci_bus 0000:08: resource 3 [io  0x0000-0xefff]
pci_bus 0000:08: resource 4 [mem 0xd0000000-0xfbffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP reno registered
UDP hash table entries: 4096 (order: 5, 131072 bytes)
UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
NET: Registered protocol family 1
pci 0000:02:00.0: Boot video device
PCI: CLS 64 bytes, default 64
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 14293k freed
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Placing 64MB software IO TLB between ffff880020000000 - ffff880024000000
software IO TLB at phys 0x20000000 - 0x24000000
microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xf
microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xf
microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xf
microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xf
microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xf
microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xf
microcode: CPU6 sig=0x106a4, pf=0x2, revision=0xf
microcode: CPU7 sig=0x106a4, pf=0x2, revision=0xf
microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
audit: initializing netlink socket (disabled)
type=2000 audit(1263338700.872:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
fuse init (API version 7.13)
msgmni has been set to 11943
SELinux:  Registering netfilter hooks
alg: No test for cipher_null (cipher_null-generic)
alg: No test for ecb(cipher_null) (ecb-cipher_null)
alg: No test for digest_null (digest_null-generic)
alg: No test for compress_null (compress_null-generic)
alg: No test for fcrypt (fcrypt-generic)
alg: No test for stdrng (krng)
alg: No test for fips(ansi_cprng) (fips_ansi_cprng)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
io scheduler noop registered (default)
io scheduler cfq registered
pcieport 0000:00:01.0: setting latency timer to 64
  alloc irq_desc for 24 on node -1
  alloc kstat_irqs on node -1
pcieport 0000:00:01.0: irq 24 for MSI/MSI-X
pcieport 0000:00:03.0: setting latency timer to 64
  alloc irq_desc for 25 on node -1
  alloc kstat_irqs on node -1
pcieport 0000:00:03.0: irq 25 for MSI/MSI-X
pcieport 0000:00:07.0: setting latency timer to 64
  alloc irq_desc for 26 on node -1
  alloc kstat_irqs on node -1
pcieport 0000:00:07.0: irq 26 for MSI/MSI-X
pcieport 0000:00:1c.0: setting latency timer to 64
  alloc irq_desc for 27 on node -1
  alloc kstat_irqs on node -1
pcieport 0000:00:1c.0: irq 27 for MSI/MSI-X
pcieport 0000:00:1c.1: setting latency timer to 64
  alloc irq_desc for 28 on node -1
  alloc kstat_irqs on node -1
pcieport 0000:00:1c.1: irq 28 for MSI/MSI-X
pcieport 0000:00:1c.4: setting latency timer to 64
  alloc irq_desc for 29 on node -1
  alloc kstat_irqs on node -1
pcieport 0000:00:1c.4: irq 29 for MSI/MSI-X
pcieport 0000:00:1c.5: setting latency timer to 64
  alloc irq_desc for 30 on node -1
  alloc kstat_irqs on node -1
pcieport 0000:00:1c.5: irq 30 for MSI/MSI-X
aer 0000:00:01.0:pcie02: AER service couldn't init device: no _OSC support
aer 0000:00:03.0:pcie02: AER service couldn't init device: no _OSC support
aer 0000:00:07.0:pcie02: AER service couldn't init device: no _OSC support
pci-stub: invalid id string ""
vesafb: framebuffer at 0xd0000000, mapped to 0xffffc90011880000, using 10240k, total 262144k
vesafb: mode is 1280x1024x32, linelength=5120, pages=0
vesafb: scrolling: redraw
vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
Console: switching to colour frame buffer device 160x64
fb0: VESA VGA frame buffer device
vga16fb: initializing
vga16fb: mapped to 0xffff8800000a0000
fb1: VGA16 VGA frame buffer device
input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
ACPI: Power Button [PWRF]
XENFS: not registering filesystem on non-xen platform
HDLC line discipline maxframe=4096
N_HDLC line discipline registered.
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
ipmi message handler version 39.2
ipmi device interface
IPMI System Interface driver.
ipmi_si: Unable to find any System Interface(s)
IPMI Watchdog: driver initialized
Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot.
brd: module loaded
loop: module loaded
ahci 0000:05:00.0: version 3.0
ahci 0000:05:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
ahci 0000:05:00.0: AHCI 0001.0000 32 slots 2 ports 3 Gbps 0x3 impl SATA mode
ahci 0000:05:00.0: flags: 64bit ncq pm led clo pmp pio slum part 
ahci 0000:05:00.0: setting latency timer to 64
scsi0 : ahci
scsi1 : ahci
ata1: SATA max UDMA/133 abar m8192@0xfbcfe000 port 0xfbcfe100 irq 17
ata2: SATA max UDMA/133 abar m8192@0xfbcfe000 port 0xfbcfe180 irq 17
ata_piix 0000:00:1f.2: version 2.13
  alloc irq_desc for 19 on node -1
  alloc kstat_irqs on node -1
ata_piix 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
ata_piix 0000:00:1f.2: setting latency timer to 64
scsi2 : ata_piix
scsi3 : ata_piix
ata3: SATA max UDMA/133 cmd 0xa000 ctl 0x9c00 bmdma 0x9480 irq 19
ata4: SATA max UDMA/133 cmd 0x9880 ctl 0x9800 bmdma 0x9488 irq 19
ata_piix 0000:00:1f.5: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ata_piix 0000:00:1f.5: MAP [ P0 -- P1 -- ]
ata_piix 0000:00:1f.5: setting latency timer to 64
scsi4 : ata_piix
scsi5 : ata_piix
ata5: SATA max UDMA/133 cmd 0x9000 ctl 0x8c00 bmdma 0x8480 irq 19
ata6: SATA max UDMA/133 cmd 0x8880 ctl 0x8800 bmdma 0x8488 irq 19
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
r8169 0000:06:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
r8169 0000:06:00.0: setting latency timer to 64
  alloc irq_desc for 31 on node -1
  alloc kstat_irqs on node -1
r8169 0000:06:00.0: irq 31 for MSI/MSI-X
eth0: RTL8168c/8111c at 0xffffc900117ae000, 00:24:21:32:03:4b, XID 1c4000c0 IRQ 31
usbcore: registered new interface driver hwa-rc
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
  alloc irq_desc for 18 on node -1
  alloc kstat_irqs on node -1
ehci_hcd 0000:00:1a.7: PCI INT C -> GSI 18 (level, low) -> IRQ 18
ehci_hcd 0000:00:1a.7: setting latency timer to 64
ehci_hcd 0000:00:1a.7: EHCI Host Controller
ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1a.7: debug port 1
ehci_hcd 0000:00:1a.7: cache line size of 64 is not supported
ehci_hcd 0000:00:1a.7: irq 18, io mem 0xf8ffe000
ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 2.6.33-rc3-marco ehci_hcd
usb usb1: SerialNumber: 0000:00:1a.7
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 6 ports detected
  alloc irq_desc for 23 on node -1
  alloc kstat_irqs on node -1
ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 23 (level, low) -> IRQ 23
ehci_hcd 0000:00:1d.7: setting latency timer to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
ehci_hcd 0000:00:1d.7: debug port 1
ehci_hcd 0000:00:1d.7: cache line size of 64 is not supported
ehci_hcd 0000:00:1d.7: irq 23, io mem 0xf8ffc000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: EHCI Host Controller
usb usb2: Manufacturer: Linux 2.6.33-rc3-marco ehci_hcd
usb usb2: SerialNumber: 0000:00:1d.7
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 6 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
uhci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
uhci_hcd 0000:00:1a.0: setting latency timer to 64
uhci_hcd 0000:00:1a.0: UHCI Host Controller
uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1a.0: irq 16, io base 0x0000ac00
usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb3: Product: UHCI Host Controller
usb usb3: Manufacturer: Linux 2.6.33-rc3-marco uhci_hcd
usb usb3: SerialNumber: 0000:00:1a.0
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
  alloc irq_desc for 21 on node -1
  alloc kstat_irqs on node -1
uhci_hcd 0000:00:1a.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21
uhci_hcd 0000:00:1a.1: setting latency timer to 64
uhci_hcd 0000:00:1a.1: UHCI Host Controller
uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:1a.1: irq 21, io base 0x0000a880
usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb4: Product: UHCI Host Controller
usb usb4: Manufacturer: Linux 2.6.33-rc3-marco uhci_hcd
usb usb4: SerialNumber: 0000:00:1a.1
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
uhci_hcd 0000:00:1a.2: PCI INT D -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1a.2: setting latency timer to 64
uhci_hcd 0000:00:1a.2: UHCI Host Controller
uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:1a.2: irq 19, io base 0x0000a800
usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb5: Product: UHCI Host Controller
usb usb5: Manufacturer: Linux 2.6.33-rc3-marco uhci_hcd
usb usb5: SerialNumber: 0000:00:1a.2
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
uhci_hcd 0000:00:1d.0: setting latency timer to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6
uhci_hcd 0000:00:1d.0: irq 23, io base 0x0000a480
usb usb6: New USB device found, idVendor=1d6b, idProduct=0001
usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb6: Product: UHCI Host Controller
usb usb6: Manufacturer: Linux 2.6.33-rc3-marco uhci_hcd
usb usb6: SerialNumber: 0000:00:1d.0
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1d.1: setting latency timer to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 7
uhci_hcd 0000:00:1d.1: irq 19, io base 0x0000a400
usb usb7: New USB device found, idVendor=1d6b, idProduct=0001
usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb7: Product: UHCI Host Controller
usb usb7: Manufacturer: Linux 2.6.33-rc3-marco uhci_hcd
usb usb7: SerialNumber: 0000:00:1d.1
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
uhci_hcd 0000:00:1d.2: setting latency timer to 64
uhci_hcd 0000:00:1d.2: UHCI Host Controller
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 8
uhci_hcd 0000:00:1d.2: irq 18, io base 0x0000a080
usb usb8: New USB device found, idVendor=1d6b, idProduct=0001
usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb8: Product: UHCI Host Controller
usb usb8: Manufacturer: Linux 2.6.33-rc3-marco uhci_hcd
usb usb8: SerialNumber: 0000:00:1d.2
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 2 ports detected
usbcore: registered new interface driver wusb-cbaf
PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1
PNP: PS/2 appears to have AUX port disabled, if this is incorrect please boot with i8042.nopnp
serio: i8042 KBD port at 0x60,0x64 irq 1
mice: PS/2 mouse device common for all mice
I2O subsystem v1.325
i2o: max drivers = 8
I2O Configuration OSM v1.323
I2O Bus Adapter OSM v1.317
I2O Block Device OSM v1.325
I2O SCSI Peripheral OSM v1.316
I2O ProcFS OSM v1.316
i2c /dev entries driver
i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
cpuidle: using governor ladder
cpuidle: using governor menu
ioatdma: Intel(R) QuickData Technology Driver 4.00
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
Advanced Linux Sound Architecture Driver Version 1.0.21.
input: PC Speaker as /devices/platform/pcspkr/input/input3
PCSP: Timer resolution is not sufficient (999848nS)
PCSP: Make sure you have HPET and ACPI enabled.
PCSP: Turned into nopcm mode.
  alloc irq_desc for 22 on node -1
  alloc kstat_irqs on node -1
HDA Intel 0000:00:1b.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
  alloc irq_desc for 32 on node -1
  alloc kstat_irqs on node -1
HDA Intel 0000:00:1b.0: irq 32 for MSI/MSI-X
HDA Intel 0000:00:1b.0: setting latency timer to 64
hda_codec: ALC889: BIOS auto-probing.
ALSA sound/pci/hda/hda_codec.c:4284: autoconfig: line_outs=4 (0x14/0x15/0x16/0x17/0x0)
ALSA sound/pci/hda/hda_codec.c:4288:    speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
ALSA sound/pci/hda/hda_codec.c:4292:    hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
ALSA sound/pci/hda/hda_codec.c:4293:    mono: mono_out=0x0
ALSA sound/pci/hda/hda_codec.c:4296:    dig-out=0x1e/0x0
ALSA sound/pci/hda/hda_codec.c:4304:    inputs: mic=0x18, fmic=0x19, line=0x1a, fline=0x0, cd=0x0, aux=0x0
ALSA sound/pci/hda/patch_realtek.c:1284: realtek: No valid SSID, checking pincfg 0x4007f603 for NID 0x1d
ALSA sound/pci/hda/patch_realtek.c:1300: realtek: Enabling init ASM_ID=0xf603 CODEC_ID=10ec0889
input: HDA Digital PCBeep as /devices/pci0000:00/0000:00:1b.0/input/input4
ALSA device list:
  #0: Dummy 1
  #1: Internal PC-Speaker at port 0x61
  #2: HDA Intel at 0xf8ff8000 irq 32
IPv4 over IPv4 tunneling driver
arp_tables: (C) 2002 David S. Miller
TCP bic registered
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
NET: Registered protocol family 15
Bridge firewalling registered
Ebtables v2.0 registered
PM: Resume from disk failed.
No TPM chip found, activating TPM-bypass!
  Magic number: 10:684:453
BIOS EDD facility v0.16 2004-Jun-25, 1 devices found
ata2: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata6: SATA link down (SStatus 0 SControl 300)
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.00: ATAPI: TSSTcorp CDDVDW SH-S223F, SB03, max UDMA/100, ATAPI AN
ata1.00: configured for UDMA/100
scsi 0:0:0:0: CD-ROM            TSSTcorp CDDVDW SH-S223F  SB03 PQ: 0 ANSI: 5
sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw xa/form2 cdda tray
Uniform CD-ROM driver Revision: 3.20
sr 0:0:0:0: Attached scsi CD-ROM sr0
sr 0:0:0:0: Attached scsi generic sg0 type 5
usb 5-1: new low speed USB device using uhci_hcd and address 2
ata3.00: SATA link down (SStatus 0 SControl 300)
ata3.01: SATA link down (SStatus 0 SControl 300)
usb 5-1: New USB device found, idVendor=046d, idProduct=c01a
usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 5-1: Product: USB-PS/2 Optical Mouse
usb 5-1: Manufacturer: Logitech
input: Logitech USB-PS/2 Optical Mouse as /devices/pci0000:00/0000:00:1a.2/usb5/5-1/5-1:1.0/input/input5
generic-usb 0003:046D:C01A.0001: input,hidraw0: USB HID v1.10 Mouse [Logitech USB-PS/2 Optical Mouse] on usb-0000:00:1a.2-1/input0
ata4.00: SATA link down (SStatus 0 SControl 300)
ata4.01: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata4.01: ATA-7: INTEL SSDSA2MH080G1GC, 045C8610, max UDMA/133
ata4.01: 156301488 sectors, multi 16: LBA48 NCQ (depth 0/31)
ata4.01: configured for UDMA/133
scsi 3:0:1:0: Direct-Access     ATA      INTEL SSDSA2MH08 045C PQ: 0 ANSI: 5
sd 3:0:1:0: [sda] 156301488 512-byte logical blocks: (80.0 GB/74.5 GiB)
sd 3:0:1:0: Attached scsi generic sg1 type 0
sd 3:0:1:0: [sda] Write Protect is off
sd 3:0:1:0: [sda] Mode Sense: 00 3a 00 00
sd 3:0:1:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2 sda3 sda4
sd 3:0:1:0: [sda] Attached SCSI disk
Freeing unused kernel memory: 1564k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 1200k freed
Freeing unused kernel memory: 1828k freed
dracut: dracut-003-1.fc13
udev: starting version 147
dracut: Starting plymouth daemon
EXT4-fs (sda3): mounted filesystem with ordered data mode
dracut: Mounted root filesystem /dev/sda3
dracut: Switching root
udev: starting version 147
rtc_cmos 00:03: RTC can wake from S4
rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
EXT4-fs (sda1): mounted filesystem with ordered data mode
Btrfs loaded
device fsid b34725b9c7d023e4-80e1a410a4178b90 devid 1 transid 9442 /dev/sda2
Btrfs detected SSD devices, enabling SSD mode
Adding 3167804k swap on /dev/sda4.  Priority:-1 extents:1 across:3167804k SS
microcode: CPU0 updated to revision 0x11, date = 2009-04-21 
microcode: CPU1 updated to revision 0x11, date = 2009-04-21 
microcode: CPU2 updated to revision 0x11, date = 2009-04-21 
microcode: CPU3 updated to revision 0x11, date = 2009-04-21 
microcode: CPU4 updated to revision 0x11, date = 2009-04-21 
microcode: CPU5 updated to revision 0x11, date = 2009-04-21 
microcode: CPU6 updated to revision 0x11, date = 2009-04-21 
microcode: CPU7 updated to revision 0x11, date = 2009-04-21 
NET: Registered protocol family 10
r8169: eth0: link up
r8169: eth0: link up
coretemp coretemp.0: Unable to access MSR 0xEE, for Tjmax, left at default
coretemp coretemp.1: Unable to access MSR 0xEE, for Tjmax, left at default
coretemp coretemp.2: Unable to access MSR 0xEE, for Tjmax, left at default
coretemp coretemp.3: Unable to access MSR 0xEE, for Tjmax, left at default
coretemp coretemp.4: Unable to access MSR 0xEE, for Tjmax, left at default
coretemp coretemp.5: Unable to access MSR 0xEE, for Tjmax, left at default
coretemp coretemp.6: Unable to access MSR 0xEE, for Tjmax, left at default
coretemp coretemp.7: Unable to access MSR 0xEE, for Tjmax, left at default
virbr0: starting userspace STP failed, starting kernel STP
ip_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
nf_conntrack.acct=1 kernel parameter, acct=1 nf_conntrack module option or
sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
Process 240(plymouthd) has RLIMIT_CORE set to 0
Aborting core
eth0: no IPv6 routers present

Re: [PATCH] clockevent: don't remove broadcast device when cpu is dead

[Xiaotian Feng]

On 01/12/2010 09:20 PM, Thomas Gleixner wrote:
> On Thu, 7 Jan 2010, Xiaotian Feng wrote:
>
>> Marc reported BUG during shutdown, after debugging, kernel is trying
>> to remove a broadcast device which mode is CLOCK_EVT_MODE_ONESHOT.
>>
>> The root cause for this bug is that in clockevents_notify,
>> "cpumask_weight(dev->cpumask) == 1" is always true even if dev is a
>
> Why is cpumask_weight(dev->cpumask) == 1 always true when we shutdown
> a non boot cpu ?
>
> The broadcast device is not a per cpu device and the cpumask should
> not only contain the CPU which is shut down !

At least for hpet broadcast dev, it's dev->cpumask is only contain the 
CPU which it is initialized from.
And for broadcast device, kernel is using tick_broadcast_mask not 
dev->cpumask, right?

>
> The patch is papering over the real problem.
>
> Marc, can you please apply the following debug patch and provide the
> dmesg outputs from boot and shutdown ?
>
> Thanks,
>
> 	tglx
> ---
>
> Index: linux-2.6-tip/kernel/time/clockevents.c
> ===================================================================
> --- linux-2.6-tip.orig/kernel/time/clockevents.c
> +++ linux-2.6-tip/kernel/time/clockevents.c
> @@ -186,7 +186,7 @@ void clockevents_register_device(struct
>   	BUG_ON(!dev->cpumask);
>
>   	raw_spin_lock_irqsave(&clockevents_lock, flags);
> -
> +	printk(KERN_ERR "CE register %p %s\n", dev, dev->name);
>   	list_add(&dev->list,&clockevent_devices);
>   	clockevents_do_notify(CLOCK_EVT_NOTIFY_ADD, dev);
>   	clockevents_notify_released();
> @@ -220,6 +220,7 @@ void clockevents_exchange_device(struct
>   	 * released list and do a notify add later.
>   	 */
>   	if (old) {
> +		printk(KERN_INFO "CE Release %p %s\n", old, old->name);
>   		clockevents_set_mode(old, CLOCK_EVT_MODE_UNUSED);
>   		list_del(&old->list);
>   		list_add(&old->list,&clockevents_released);
> @@ -260,6 +261,13 @@ void clockevents_notify(unsigned long re
>   		list_for_each_entry_safe(dev, tmp,&clockevent_devices, list) {
>   			if (cpumask_test_cpu(cpu, dev->cpumask)&&
>   			cpumask_weight(dev->cpumask) == 1) {
> +				if (dev->mode != CLOCK_EVT_MODE_UNUSED) {
> +					printk(KERN_INFO
> +					       "CE Remove %p %s bc: %d\n",
> +					       dev, dev->name,
> +					       tick_is_broadcast_device(dev));
> +					continue;
> +				}
>   				BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
>   				list_del(&dev->list);
>   			}
> Index: linux-2.6-tip/kernel/time/tick-broadcast.c
> ===================================================================
> --- linux-2.6-tip.orig/kernel/time/tick-broadcast.c
> +++ linux-2.6-tip/kernel/time/tick-broadcast.c
> @@ -72,6 +72,8 @@ int tick_check_broadcast_device(struct c
>   	     (dev->features&  CLOCK_EVT_FEAT_C3STOP))
>   		return 0;
>
> +	printk(KERN_INFO "CE set broadcast %p %s\n", dev, dev->name);
> +
>   	clockevents_exchange_device(NULL, dev);
>   	tick_broadcast_device.evtdev = dev;
>   	if (!cpumask_empty(tick_get_broadcast_mask()))
>

Re: [PATCH] clockevent: don't remove broadcast device when cpu is dead

[Thomas Gleixner]

On Wed, 13 Jan 2010, Xiaotian Feng wrote:
> On 01/12/2010 09:20 PM, Thomas Gleixner wrote:
> > On Thu, 7 Jan 2010, Xiaotian Feng wrote:
> > 
> > > Marc reported BUG during shutdown, after debugging, kernel is trying
> > > to remove a broadcast device which mode is CLOCK_EVT_MODE_ONESHOT.
> > > 
> > > The root cause for this bug is that in clockevents_notify,
> > > "cpumask_weight(dev->cpumask) == 1" is always true even if dev is a
> > 
> > Why is cpumask_weight(dev->cpumask) == 1 always true when we shutdown
> > a non boot cpu ?
> > 
> > The broadcast device is not a per cpu device and the cpumask should
> > not only contain the CPU which is shut down !
> 
> At least for hpet broadcast dev, it's dev->cpumask is only contain the CPU
> which it is initialized from.

Which is fundamentaly wrong and the root cause of the problem. I'll
have a look tomorrow morning when my brain is more awake than now.

> And for broadcast device, kernel is using tick_broadcast_mask not
> dev->cpumask, right?

No, tick_broadcast_mask is the bitmask which tells us which cpus get
the broadcast IPI.

Thanks,

	tglx

Re: [PATCH] clockevent: don't remove broadcast device when cpu is dead

[Xiaotian Feng]

On 01/14/2010 06:08 AM, Thomas Gleixner wrote:
> On Wed, 13 Jan 2010, Xiaotian Feng wrote:
>> On 01/12/2010 09:20 PM, Thomas Gleixner wrote:
>>> On Thu, 7 Jan 2010, Xiaotian Feng wrote:
>>>
>>>> Marc reported BUG during shutdown, after debugging, kernel is trying
>>>> to remove a broadcast device which mode is CLOCK_EVT_MODE_ONESHOT.
>>>>
>>>> The root cause for this bug is that in clockevents_notify,
>>>> "cpumask_weight(dev->cpumask) == 1" is always true even if dev is a
>>>
>>> Why is cpumask_weight(dev->cpumask) == 1 always true when we shutdown
>>> a non boot cpu ?
>>>
>>> The broadcast device is not a per cpu device and the cpumask should
>>> not only contain the CPU which is shut down !
>>
>> At least for hpet broadcast dev, it's dev->cpumask is only contain the CPU
>> which it is initialized from.
>
> Which is fundamentaly wrong and the root cause of the problem. I'll
> have a look tomorrow morning when my brain is more awake than now.

hpet_legacy_clockevent_register is trying to register new CE, but 
replace failed,
then in tick_check_new_device -> tick_check_broadcast_device, the legacy 
hpet CE
was registered as multicast device, but its dev->cpumask is cpumask of 
smp_processor_id().

on my system its dev->cpumask is cpumask of 0, but in Marc's, 
dev->cpumask is cpumask of 4.
So when kernel is trying to offline cpu 4, the broadcast hpet is removed.

>
>> And for broadcast device, kernel is using tick_broadcast_mask not
>> dev->cpumask, right?
>
> No, tick_broadcast_mask is the bitmask which tells us which cpus get
> the broadcast IPI.
>
> Thanks,
>
> 	tglx
>

Re: [PATCH] clockevent: don't remove broadcast device when cpu is dead

[Thomas Gleixner]

On Thu, 14 Jan 2010, Xiaotian Feng wrote:
> On 01/14/2010 06:08 AM, Thomas Gleixner wrote:
> > On Wed, 13 Jan 2010, Xiaotian Feng wrote:
> > > On 01/12/2010 09:20 PM, Thomas Gleixner wrote:
> > > > On Thu, 7 Jan 2010, Xiaotian Feng wrote:
> > > > 
> > > > > Marc reported BUG during shutdown, after debugging, kernel is trying
> > > > > to remove a broadcast device which mode is CLOCK_EVT_MODE_ONESHOT.
> > > > > 
> > > > > The root cause for this bug is that in clockevents_notify,
> > > > > "cpumask_weight(dev->cpumask) == 1" is always true even if dev is a
> > > > 
> > > > Why is cpumask_weight(dev->cpumask) == 1 always true when we shutdown
> > > > a non boot cpu ?
> > > > 
> > > > The broadcast device is not a per cpu device and the cpumask should
> > > > not only contain the CPU which is shut down !
> > > 
> > > At least for hpet broadcast dev, it's dev->cpumask is only contain the CPU
> > > which it is initialized from.
> > 
> > Which is fundamentaly wrong and the root cause of the problem. I'll
> > have a look tomorrow morning when my brain is more awake than now.
> 
> hpet_legacy_clockevent_register is trying to register new CE, but replace
> failed,
> then in tick_check_new_device -> tick_check_broadcast_device, the legacy hpet
> CE
> was registered as multicast device, but its dev->cpumask is cpumask of
> smp_processor_id().
> 
> on my system its dev->cpumask is cpumask of 0, but in Marc's, dev->cpumask is
> cpumask of 4.
> So when kernel is trying to offline cpu 4, the broadcast hpet is removed.

I know, but the point is that a broadcast device should not be bound
to the CPU which is registering it. Working on a fix.

Thanks,

	tglx

[tip:timers/urgent] clockevent: Don't remove broadcast device when cpu is dead

[tip-bot for Xiaotian Feng]

Commit-ID:  ea9d8e3f45404d411c00ae67b45cc35c58265bb7
Gitweb:     http://git.kernel.org/tip/ea9d8e3f45404d411c00ae67b45cc35c58265bb7
Author:     Xiaotian Feng <dfeng@redhat.com>
AuthorDate: Thu, 7 Jan 2010 11:22:44 +0800
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Mon, 18 Jan 2010 14:44:50 +0100

clockevent: Don't remove broadcast device when cpu is dead

Marc reported that the BUG_ON in clockevents_notify() triggers on his
system. This happens because the kernel tries to remove an active
clock event device (used for broadcasting) from the device list.

The handling of devices which can be used as per cpu device and as a
global broadcast device is suboptimal.

The simplest solution for now (and for stable) is to check whether the
device is used as global broadcast device, but this needs to be
revisited.

[ tglx: restored the cpuweight check and massaged the changelog ]

Reported-by: Marc Dionne <marc.c.dionne@gmail.com>
Tested-by: Marc Dionne <marc.c.dionne@gmail.com>
Signed-off-by: Xiaotian Feng <dfeng@redhat.com>
LKML-Reference: <1262834564-13033-1-git-send-email-dfeng@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@kernel.org
---
 kernel/time/clockevents.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
index 6f740d9..d7395fd 100644
--- a/kernel/time/clockevents.c
+++ b/kernel/time/clockevents.c
@@ -259,7 +259,8 @@ void clockevents_notify(unsigned long reason, void *arg)
 		cpu = *((int *)arg);
 		list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) {
 			if (cpumask_test_cpu(cpu, dev->cpumask) &&
-			    cpumask_weight(dev->cpumask) == 1) {
+			    cpumask_weight(dev->cpumask) == 1 &&
+			    !tick_is_broadcast_device(dev)) {
 				BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
 				list_del(&dev->list);
 			}