More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2175 bytes --]

Hello,

After upgrading to 2.6.16-rc6 I noticed this strange message:

More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.

This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with 
totoal of 4 logical CPUs).

#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
CONFIG_MPENTIUM4=y
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_SMP=y
CONFIG_NR_CPUS=4
CONFIG_SCHED_SMT=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_NONFATAL=y
CONFIG_X86_MCE_P4THERMAL=y
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_X86_REBOOTFIXUPS is not set
CONFIG_MICROCODE=y
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set


Best regards,

 			Krzysztof Olędzki

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Andrew Morton]

Krzysztof Oledzki <olel@ans.pl> wrote:
>
> After upgrading to 2.6.16-rc6 I noticed this strange message:
> 
>  More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
>  Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
>
> This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with 
>  totoal of 4 logical CPUs).

Please send full dmesg output for the failing kernel, thanks.

Which is the most-recently-tested kernel which behaved correctly?

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 603 bytes --]



On Sat, 11 Mar 2006, Andrew Morton wrote:

> Krzysztof Oledzki <olel@ans.pl> wrote:
>>
>> After upgrading to 2.6.16-rc6 I noticed this strange message:
>>
>>  More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
>>  Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
>>
>> This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with
>>  totoal of 4 logical CPUs).
>
> Please send full dmesg output for the failing kernel, thanks.
Attached.

> Which is the most-recently-tested kernel which behaved correctly?
2.6.15.6

Best regards,

 					Krzysztof Olędzki

[-- Attachment #2: Type: TEXT/PLAIN, Size: 23230 bytes --]

Linux version 2.6.16-rc6 (root@fw2) (gcc version 3.4.5) #1 SMP PREEMPT Sun Mar 12 02:43:26 CET 2006
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
 BIOS-e820: 0000000000100000 - 000000007ffc0000 (usable)
 BIOS-e820: 000000007ffc0000 - 000000007ffcfc00 (ACPI data)
 BIOS-e820: 000000007ffcfc00 - 000000007ffff000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec90000 (reserved)
 BIOS-e820: 00000000fed00000 - 00000000fed00400 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
 BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved)
2047MB LOWMEM available.
found SMP MP-table at 000fe710
On node 0 totalpages: 524224
  DMA zone: 4096 pages, LIFO batch:0
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 520128 pages, LIFO batch:31
  HighMem zone: 0 pages, LIFO batch:0
DMI 2.3 present.
ACPI: RSDP (v000 DELL                                  ) @ 0x000fd650
ACPI: RSDT (v001 DELL   PESC1425 0x00000001 MSFT 0x0100000a) @ 0x000fd664
ACPI: FADT (v001 DELL   PESC1425 0x00000001 MSFT 0x0100000a) @ 0x000fd6b0
ACPI: MADT (v001 DELL   PESC1425 0x00000001 MSFT 0x0100000a) @ 0x000fd724
ACPI: SPCR (v001 DELL   PESC1425 0x00000001 MSFT 0x0100000a) @ 0x000fd7c0
ACPI: HPET (v001 DELL   PESC1425 0x00000001 MSFT 0x0100000a) @ 0x000fd810
ACPI: MCFG (v001 DELL   PESC1425 0x00000001 MSFT 0x0100000a) @ 0x000fd848
ACPI: DSDT (v001 DELL   PESC1425 0x00000001 MSFT 0x0100000e) @ 0x00000000
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x06] enabled)
Processor #6 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
Processor #1 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] enabled)
Processor #7 15:4 APIC version 20
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x09] address[0xfec80000] gsi_base[32])
IOAPIC[1]: apic_id 9, version 32, address 0xfec80000, GSI 32-55
ACPI: IOAPIC (id[0x0a] address[0xfec80800] gsi_base[64])
IOAPIC[2]: apic_id 10, version 32, address 0xfec80800, GSI 64-87
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Enabling APIC mode:  Flat.  Using 3 I/O APICs
ACPI: HPET id: 0xffffffff base: 0xfed00000
More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 80000000 (gap: 7ffff000:60001000)
Built 1 zonelists
Kernel command line: auto BOOT_IMAGE=Linux-2.6.16r6 ro root=900 rootflags=data=journal ip_conntrack.hashsize=131072
mapped APIC to ffffd000 (fee00000)
mapped IOAPIC to ffffc000 (fec00000)
mapped IOAPIC to ffffb000 (fec80000)
mapped IOAPIC to ffffa000 (fec80800)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 65536 bytes)
Console: colour VGA+ 80x30
Dentry cache hash table entries: 524288 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 262144 (order: 8, 1048576 bytes)
Memory: 2072088k/2096896k available (2967k kernel code, 24404k reserved, 1041k data, 216k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
hpet0: at MMIO 0xfed00000 (virtual 0xf8800000), IRQs 2, 8, 0
hpet0: 3 64-bit timers, 14318180 Hz
Using HPET for base-timer
Using HPET for gettimeofday
Detected 3200.687 MHz processor.
Using hpet for high-res timesource
Calibrating delay using timer specific routine.. 6404.87 BogoMIPS (lpj=3202439)
Mount-cache hash table entries: 512
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
using mwait in idle threads.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 20100000 00000000 00000180 0000641d 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (24) available
CPU0: Thermal monitoring enabled
Checking 'hlt' instruction... OK.
CPU0: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
Booting processor 1/1 eip 3000
Initializing CPU#1
Calibrating delay using timer specific routine.. 6400.28 BogoMIPS (lpj=3200140)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 20100000 00000000 00000180 0000641d 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#1.
CPU1: Intel P4/Xeon Extended MCE MSRs (24) available
CPU1: Thermal monitoring enabled
CPU1: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
Booting processor 2/6 eip 3000
Initializing CPU#2
Calibrating delay using timer specific routine.. 6400.31 BogoMIPS (lpj=3200157)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 3
CPU: After all inits, caps: bfebfbff 20100000 00000000 00000180 0000641d 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#2.
CPU2: Intel P4/Xeon Extended MCE MSRs (24) available
CPU2: Thermal monitoring enabled
CPU2: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
Booting processor 3/7 eip 3000
Initializing CPU#3
Calibrating delay using timer specific routine.. 6400.32 BogoMIPS (lpj=3200161)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 3
CPU: After all inits, caps: bfebfbff 20100000 00000000 00000180 0000641d 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#3.
CPU3: Intel P4/Xeon Extended MCE MSRs (24) available
CPU3: Thermal monitoring enabled
CPU3: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
Total of 4 processors activated (25605.79 BogoMIPS).
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1
checking TSC synchronization across 4 CPUs: passed.
Brought up 4 CPUs
migration_cost=1000,1000
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfc3de, last bus=4
PCI: Using MMCONFIG
ACPI: Subsystem revision 20060127
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: Probing PCI hardware (bus 00)
PCI quirk: region 0800-087f claimed by ICH4 ACPI/GPIO/TCO
PCI quirk: region 0880-08bf claimed by ICH4 GPIO
PCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1
PCI: PXH quirk detected, disabling MSI for SHPC device
PCI: PXH quirk detected, disabling MSI for SHPC device
Boot video device is 0000:04:0d.0
PCI: Transparent bridge - 0000:00:1e.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PALO._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PALO.PXHB._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PALO.PXHA._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PICH._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 10 *11 12)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 7 10 11 12)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 *10 11 12)
ACPI: PCI Interrupt Link [LNKE] (IRQs *3 4 5 6 7 10 11 12)
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12) *0, disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 *6 7 10 11 12)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
pnp: PnP ACPI: found 11 devices
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
TC classifier action (bugs to netdev@vger.kernel.org cc hadi@cyberus.ca)
pnp: 00:08: ioport range 0x800-0x87f could not be reserved
pnp: 00:08: ioport range 0x880-0x8bf has been reserved
pnp: 00:08: ioport range 0x8c0-0x8df has been reserved
pnp: 00:08: ioport range 0x8e0-0x8e3 has been reserved
pnp: 00:08: ioport range 0xc00-0xc0f has been reserved
pnp: 00:08: ioport range 0xc10-0xc1f has been reserved
pnp: 00:08: ioport range 0xca0-0xcaf has been reserved
pnp: 00:08: ioport range 0xc20-0xc3f has been reserved
PCI: Bridge: 0000:01:00.0
  IO window: e000-efff
  MEM window: fe900000-feafffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:01:00.2
  IO window: disabled.
  MEM window: disabled.
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:02.0
  IO window: e000-efff
  MEM window: fe700000-feafffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:1e.0
  IO window: d000-dfff
  MEM window: fe500000-fe6fffff
  PREFETCH window: f0000000-f7ffffff
ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 169
PCI: Setting latency timer of device 0000:00:02.0 to 64
PCI: Setting latency timer of device 0000:01:00.0 to 64
PCI: Setting latency timer of device 0000:01:00.2 to 64
PCI: Setting latency timer of device 0000:00:1e.0 to 64
Machine check exception polling timer started.
IA-32 Microcode Update Driver: v1.14 <tigran@veritas.com>
audit: initializing netlink socket (disabled)
audit(1142131816.949:1): initialized
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered (default)
io scheduler cfq registered
Intel E7520/7320/7525 detected.<6>ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 169
PCI: Setting latency timer of device 0000:00:02.0 to 64
Allocate Port Service[0000:00:02.0:pcie00]
Allocate Port Service[0000:00:02.0:pcie01]
ACPI: Power Button (FF) [PWRF]
ACPI: Video Device [EVGA] (multi-head: no  rom: yes  post: no)
Real Time Clock Driver v1.12ac
hpet_resources: 0xfed00000 is busy
ipmi message handler version 38.0
ipmi device interface
IPMI System Interface driver.
ipmi_si: Found SMBIOS-specified state machine at I/O address 0xca8, slave address 0x20
 IPMI kcs interface initialized
IPMI Watchdog: driver initialized
Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot.
IPMI poweroff: Found a chassis style poweroff function
PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:07: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
floppy0: no floppy controllers found
loop: loaded (max 8 devices)
Intel(R) PRO/1000 Network Driver - version 6.3.9-k4-NAPI
Copyright (c) 1999-2005 Intel Corporation.
ACPI: PCI Interrupt 0000:02:04.0[A] -> GSI 32 (level, low) -> IRQ 177
e1000: 0000:02:04.0: e1000_probe: (PCI:66MHz:32-bit) 00:14:22:b0:cb:52
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:04:03.0[A] -> GSI 20 (level, low) -> IRQ 185
e1000: 0000:04:03.0: e1000_probe: (PCI:33MHz:32-bit) 00:14:22:b0:cb:53
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ICH5: IDE controller at PCI slot 0000:00:1f.1
PCI: Enabling device 0000:00:1f.1 (0005 -> 0007)
ACPI: PCI Interrupt 0000:00:1f.1[A] -> GSI 18 (level, low) -> IRQ 193
ICH5: chipset revision 2
ICH5: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xfc00-0xfc07, BIOS settings: hda:DMA, hdb:pio
Probing IDE interface ide0...
hda: HL-DT-ST DVD-ROM GDR-8084N, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hda: ATAPI 24X DVD-ROM drive, 2048kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
libata version 1.20 loaded.
ata_piix 0000:00:1f.2: version 1.05
ACPI: PCI Interrupt 0000:00:1f.2[A] -> GSI 18 (level, low) -> IRQ 193
PCI: Setting latency timer of device 0000:00:1f.2 to 64
ata1: SATA max UDMA/133 cmd 0xCCB8 ctl 0xCCB2 bmdma 0xCC80 irq 193
ata2: SATA max UDMA/133 cmd 0xCCA0 ctl 0xCC9A bmdma 0xCC88 irq 193
ata1: dev 0 cfg 49:2f00 82:746b 83:7f01 84:4023 85:7469 86:3c01 87:4023 88:207f
ata1: dev 0 ATA-7, max UDMA/133, 156250000 sectors: LBA48
ata1: dev 0 configured for UDMA/133
scsi0 : ata_piix
ata2: dev 0 cfg 49:2f00 82:746b 83:7f01 84:4023 85:7469 86:3c01 87:4023 88:207f
ata2: dev 0 ATA-7, max UDMA/133, 156250000 sectors: LBA48
ata2: dev 0 configured for UDMA/133
scsi1 : ata_piix
  Vendor: ATA       Model: WDC WD800JD-75MS  Rev: 10.0
  Type:   Direct-Access                      ANSI SCSI revision: 05
  Vendor: ATA       Model: WDC WD800JD-75MS  Rev: 10.0
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sda: 156250000 512-byte hdwr sectors (80000 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
SCSI device sda: 156250000 512-byte hdwr sectors (80000 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
 sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 sda8 sda9 >
sd 0:0:0:0: Attached scsi disk sda
SCSI device sdb: 156250000 512-byte hdwr sectors (80000 MB)
sdb: Write Protect is off
sdb: Mode Sense: 00 3a 00 00
SCSI device sdb: drive cache: write back
SCSI device sdb: 156250000 512-byte hdwr sectors (80000 MB)
sdb: Write Protect is off
sdb: Mode Sense: 00 3a 00 00
SCSI device sdb: drive cache: write back
 sdb: sdb1 sdb2 sdb3 sdb4 < sdb5 sdb6 sdb7 sdb8 sdb9 >
sd 1:0:0:0: Attached scsi disk sdb
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 1:0:0:0: Attached scsi generic sg1 type 0
ACPI: PCI Interrupt 0000:00:1d.7[D] -> GSI 23 (level, low) -> IRQ 201
PCI: Setting latency timer of device 0000:00:1d.7 to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: debug port 1
PCI: cache line size of 128 is not supported by device 0000:00:1d.7
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1d.7: irq 201, io mem 0xfeb00000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 4 ports detected
USB Universal Host Controller Interface driver v2.3
ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 16 (level, low) -> IRQ 169
PCI: Setting latency timer of device 0000:00:1d.0 to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
uhci_hcd 0000:00:1d.0: irq 169, io base 0x0000cce0
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) -> IRQ 209
PCI: Setting latency timer of device 0000:00:1d.1 to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1d.1: irq 209, io base 0x0000ccc0
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
usbcore: registered new driver usblp
drivers/usb/class/usblp.c: v0.13: USB Printer Device Class driver
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard as /class/input/input0
input: PC Speaker as /class/input/input1
md: raid1 personality registered for level 1
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
 dcdbas: Dell Systems Management Base Driver (version 5.6.0-2)
GACT probability on
Mirror/redirect action on
Simple TC action Loaded
netem: version 1.2
u32 classifier
    Perfomance counters on
    input device check on 
    Actions configured 
Netfilter messages via NETLINK v0.30.
NET: Registered protocol family 2
IP route cache hash table entries: 131072 (order: 7, 524288 bytes)
TCP established hash table entries: 262144 (order: 9, 3145728 bytes)
TCP bind hash table entries: 65536 (order: 7, 786432 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
IPv4 over IPv4 tunneling driver
GRE over IPv4 tunneling driver
ip_conntrack version 2.4 (131072 buckets, 1048576 max) - 232 bytes per conntrack
ctnetlink v0.90: registering with nfnetlink.
ip_conntrack_pptp version 3.1 loaded
ip_nat_pptp version 3.0 loaded
ip_tables: (C) 2000-2006 Netfilter Core Team
ipt_time loading
ipt_random match loaded
ipt_recent v0.3.1: Stephen Frost <sfrost@snowman.net>.  http://snowman.net/projects/ipt_recent/
IPP2P v0.8.1_rc1 loading
ClusterIP Version 0.8 loaded successfully
arp_tables: (C) 2002 David S. Miller
IPVS: Registered protocols (TCP, UDP)
IPVS: Connection hash table configured (size=65536, memory=512Kbytes)
IPVS: ipvs loaded.
IPVS: [rr] scheduler registered.
IPVS: [wrr] scheduler registered.
IPVS: [lc] scheduler registered.
IPVS: [wlc] scheduler registered.
IPVS: [lblc] scheduler registered.
IPVS: [lblcr] scheduler registered.
IPVS: [dh] scheduler registered.
IPVS: [sh] scheduler registered.
IPVS: [sed] scheduler registered.
IPVS: [nq] scheduler registered.
TCP bic registered
TCP cubic registered
TCP westwood registered
TCP highspeed registered
TCP hybla registered
TCP htcp registered
TCP vegas registered
TCP scalable registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
ip6_tables: (C) 2000-2006 Netfilter Core Team
NET: Registered protocol family 17
NET: Registered protocol family 15
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
p4-clockmod: P4/Xeon(TM) CPU On-Demand Clock Modulation available
Using IPI No-Shortcut mode
ACPI wakeup devices: 
PCI0 PALO  PXH PXHB PXHA PICH 
ACPI: (supports S0 S4 S5)
BIOS EDD facility v0.16 2004-Jun-25, 2 devices found
md: Autodetecting RAID arrays.
input: ImExPS/2 Logitech Explorer Mouse as /class/input/input2
md: autorun ...
md: considering sdb9 ...
md:  adding sdb9 ...
md: sdb8 has different UUID to sdb9
md: sdb7 has different UUID to sdb9
md: sdb6 has different UUID to sdb9
md: sdb5 has different UUID to sdb9
md: sdb3 has different UUID to sdb9
md: sdb2 has different UUID to sdb9
md:  adding sda9 ...
md: sda8 has different UUID to sdb9
md: sda7 has different UUID to sdb9
md: sda6 has different UUID to sdb9
md: sda5 has different UUID to sdb9
md: sda3 has different UUID to sdb9
md: sda2 has different UUID to sdb9
md: created md5
md: bind<sda9>
md: bind<sdb9>
md: running: <sdb9><sda9>
raid1: raid set md5 active with 2 out of 2 mirrors
md: considering sdb8 ...
md:  adding sdb8 ...
md: sdb7 has different UUID to sdb8
md: sdb6 has different UUID to sdb8
md: sdb5 has different UUID to sdb8
md: sdb3 has different UUID to sdb8
md: sdb2 has different UUID to sdb8
md:  adding sda8 ...
md: sda7 has different UUID to sdb8
md: sda6 has different UUID to sdb8
md: sda5 has different UUID to sdb8
md: sda3 has different UUID to sdb8
md: sda2 has different UUID to sdb8
md: created md4
md: bind<sda8>
md: bind<sdb8>
md: running: <sdb8><sda8>
raid1: raid set md4 active with 2 out of 2 mirrors
md: considering sdb7 ...
md:  adding sdb7 ...
md: sdb6 has different UUID to sdb7
md: sdb5 has different UUID to sdb7
md: sdb3 has different UUID to sdb7
md: sdb2 has different UUID to sdb7
md:  adding sda7 ...
md: sda6 has different UUID to sdb7
md: sda5 has different UUID to sdb7
md: sda3 has different UUID to sdb7
md: sda2 has different UUID to sdb7
md: created md3
md: bind<sda7>
md: bind<sdb7>
md: running: <sdb7><sda7>
raid1: raid set md3 active with 2 out of 2 mirrors
md: considering sdb6 ...
md:  adding sdb6 ...
md: sdb5 has different UUID to sdb6
md: sdb3 has different UUID to sdb6
md: sdb2 has different UUID to sdb6
md:  adding sda6 ...
md: sda5 has different UUID to sdb6
md: sda3 has different UUID to sdb6
md: sda2 has different UUID to sdb6
md: created md2
md: bind<sda6>
md: bind<sdb6>
md: running: <sdb6><sda6>
raid1: raid set md2 active with 2 out of 2 mirrors
md: considering sdb5 ...
md:  adding sdb5 ...
md: sdb3 has different UUID to sdb5
md: sdb2 has different UUID to sdb5
md:  adding sda5 ...
md: sda3 has different UUID to sdb5
md: sda2 has different UUID to sdb5
md: created md1
md: bind<sda5>
md: bind<sdb5>
md: running: <sdb5><sda5>
raid1: raid set md1 active with 2 out of 2 mirrors
md: considering sdb3 ...
md:  adding sdb3 ...
md: sdb2 has different UUID to sdb3
md:  adding sda3 ...
md: sda2 has different UUID to sdb3
md: created md15
md: bind<sda3>
md: bind<sdb3>
md: running: <sdb3><sda3>
raid1: raid set md15 active with 2 out of 2 mirrors
md: considering sdb2 ...
md:  adding sdb2 ...
md:  adding sda2 ...
md: created md0
md: bind<sda2>
md: bind<sdb2>
md: running: <sdb2><sda2>
raid1: raid set md0 active with 2 out of 2 mirrors
md: ... autorun DONE.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with journal data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 216k freed
Adding 2007992k swap on /dev/md15.  Priority:8192 extents:1 across:2007992k
EXT3 FS on md0, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on md1, internal journal
EXT3-fs: mounted filesystem with journal data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on md2, internal journal
EXT3-fs: mounted filesystem with journal data mode.

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Andrew Morton]

Krzysztof Oledzki <olel@ans.pl> wrote:
>
> On Sat, 11 Mar 2006, Andrew Morton wrote:
> 
>  > Krzysztof Oledzki <olel@ans.pl> wrote:
>  >>
>  >> After upgrading to 2.6.16-rc6 I noticed this strange message:
>  >>
>  >>  More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
>  >>  Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
>  >>
>  >> This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with
>  >>  totoal of 4 logical CPUs).
>  >
>  > Please send full dmesg output for the failing kernel, thanks.
>  Attached.
> 
>  > Which is the most-recently-tested kernel which behaved correctly?
>  2.6.15.6

OK, thanks.  I assume the machine's working OK?

>From my reading, you have CONFIG_HOTPLUG_CPU enabled and the machine has an
APIC.  I'd expect that lots of people would hit that warning but for some
reason they don't - possibly because most APICs don't have sufficiently
high version numbers?

Anyway, various people cc'ed.  I _think_ it's harmless, although the way in
which def_to_bigsmp propagates into the DMI and APIC code might be a
problem, depending upon config options.

Certainly the warning is incorrect, but I'm not sure what is the best thing
to do about it?

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1441 bytes --]



On Sun, 12 Mar 2006, Andrew Morton wrote:

> Krzysztof Oledzki <olel@ans.pl> wrote:
>>
>> On Sat, 11 Mar 2006, Andrew Morton wrote:
>>
>> > Krzysztof Oledzki <olel@ans.pl> wrote:
>> >>
>> >> After upgrading to 2.6.16-rc6 I noticed this strange message:
>> >>
>> >>  More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
>> >>  Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
>> >>
>> >> This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with
>> >>  totoal of 4 logical CPUs).
>> >
>> > Please send full dmesg output for the failing kernel, thanks.
>>  Attached.
>>
>> > Which is the most-recently-tested kernel which behaved correctly?
>>  2.6.15.6
>
> OK, thanks.  I assume the machine's working OK?

Yes. So far no problems, only this warning.

> From my reading, you have CONFIG_HOTPLUG_CPU enabled and the machine has an
> APIC.
That is correct.

> I'd expect that lots of people would hit that warning but for some
> reason they don't - possibly because most APICs don't have sufficiently
> high version numbers?
>
> Anyway, various people cc'ed.  I _think_ it's harmless, although the way in
> which def_to_bigsmp propagates into the DMI and APIC code might be a
> problem, depending upon config options.
>
> Certainly the warning is incorrect, but I'm not sure what is the best thing
> to do about it?

OK. Thank you.

Best regards,

 				Krzysztof Olędzki

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Venkatesh Pallipadi]

On Sun, Mar 12, 2006 at 02:05:00PM +0100, Krzysztof Oledzki wrote:
> 
> 
> On Sun, 12 Mar 2006, Andrew Morton wrote:
> 
> > Krzysztof Oledzki <olel@ans.pl> wrote:
> >>
> >> On Sat, 11 Mar 2006, Andrew Morton wrote:
> >>
> >> > Krzysztof Oledzki <olel@ans.pl> wrote:
> >> >>
> >> >> After upgrading to 2.6.16-rc6 I noticed this strange message:
> >> >>
> >> >>  More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
> >> >>  Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
> >> >>
> >> >> This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with
> >> >>  totoal of 4 logical CPUs).
> >> >
> >> > Please send full dmesg output for the failing kernel, thanks.
> >>  Attached.
> >>
> >> > Which is the most-recently-tested kernel which behaved correctly?
> >>  2.6.15.6
> >
> > OK, thanks.  I assume the machine's working OK?
> 
> Yes. So far no problems, only this warning.
> 
> > From my reading, you have CONFIG_HOTPLUG_CPU enabled and the machine has an
> > APIC.
> That is correct.
> 
> > I'd expect that lots of people would hit that warning but for some
> > reason they don't - possibly because most APICs don't have sufficiently
> > high version numbers?
> >

Actually, this warning should be seen on many other systems on well. We
use the bigsmp when there _or_ more than 8 CPUs or CPU_HOTPLUG is used.
So, in that sense the message is wrong, it should also have CPU_HOTPLUG in
there. Or we should make CPU_HOTPLUG depend on GENERIC_ARCH or auto select
GENERIC_ARCH with hotplug at the CONFIG level.

Will defer to Ashok for a proper fix.

Thanks,
Venki

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1999 bytes --]



On Sun, 12 Mar 2006, Venkatesh Pallipadi wrote:

> On Sun, Mar 12, 2006 at 02:05:00PM +0100, Krzysztof Oledzki wrote:
>>
>>
>> On Sun, 12 Mar 2006, Andrew Morton wrote:
>>
>>> Krzysztof Oledzki <olel@ans.pl> wrote:
>>>>
>>>> On Sat, 11 Mar 2006, Andrew Morton wrote:
>>>>
>>>>> Krzysztof Oledzki <olel@ans.pl> wrote:
>>>>>>
>>>>>> After upgrading to 2.6.16-rc6 I noticed this strange message:
>>>>>>
>>>>>>  More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
>>>>>>  Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
>>>>>>
>>>>>> This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with
>>>>>>  totoal of 4 logical CPUs).
>>>>>
>>>>> Please send full dmesg output for the failing kernel, thanks.
>>>>  Attached.
>>>>
>>>>> Which is the most-recently-tested kernel which behaved correctly?
>>>>  2.6.15.6
>>>
>>> OK, thanks.  I assume the machine's working OK?
>>
>> Yes. So far no problems, only this warning.
>>
>>> From my reading, you have CONFIG_HOTPLUG_CPU enabled and the machine has an
>>> APIC.
>> That is correct.
>>
>>> I'd expect that lots of people would hit that warning but for some
>>> reason they don't - possibly because most APICs don't have sufficiently
>>> high version numbers?
>>>
>
> Actually, this warning should be seen on many other systems on well. We
> use the bigsmp when there _or_ more than 8 CPUs or CPU_HOTPLUG is used.
> So, in that sense the message is wrong, it should also have CPU_HOTPLUG in
> there. Or we should make CPU_HOTPLUG depend on GENERIC_ARCH or auto select
> GENERIC_ARCH with hotplug at the CONFIG level.

Why? I have exactly 4 HT CPUs (2 cores), no more. I use CPU hotplug so I 
can disable or enable any of them when I want to. So, this is a classic 
SMP system and 2.6.15 is totally happy with this. Or is there any other 
(better?) way to disable/enable CPU (especially second logical CPU from 
HT) on running systems?

Best regards,

 				Krzysztof Olędzki

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Andrew Morton]

Krzysztof Oledzki <olel@ans.pl> wrote:
>
> > Actually, this warning should be seen on many other systems on well. We
>  > use the bigsmp when there _or_ more than 8 CPUs or CPU_HOTPLUG is used.
>  > So, in that sense the message is wrong, it should also have CPU_HOTPLUG in
>  > there. Or we should make CPU_HOTPLUG depend on GENERIC_ARCH or auto select
>  > GENERIC_ARCH with hotplug at the CONFIG level.
> 
>  Why? I have exactly 4 HT CPUs (2 cores), no more. I use CPU hotplug so I 
>  can disable or enable any of them when I want to. So, this is a classic 
>  SMP system and 2.6.15 is totally happy with this. Or is there any other 
>  (better?) way to disable/enable CPU (especially second logical CPU from 
>  HT) on running systems?

Maybe we should have:

	if (num_possible_cpus() <= 8)
		dont_do_any_of_that_stuff();

That's assuming that hotplug-cpu-capable platforms are correctly setting
cpu_possible_map.  Do they?

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Ashok Raj]

On Sun, Mar 12, 2006 at 02:30:53PM -0800, Andrew Morton wrote:
> 
> Maybe we should have:
> 
> 	if (num_possible_cpus() <= 8)
> 		dont_do_any_of_that_stuff();
> 
> That's assuming that hotplug-cpu-capable platforms are correctly setting
> cpu_possible_map.  Do they?

That wont work, since we use HOTPLUG_CPU to suspend/resume as well. We 
switched to using bigsmp (that uses physflat for IPI's) just to avoid
sending IPI's to offline CPUs. When we use logical flat we use shortcuts
that have ill effects on CPUs that are offline.

Think making CONFIG_HOTPLUG_CPU depend on X86_GENERICARCH, or X86_BIGSMP
seems like a better choice.

-- 
Cheers,
Ashok Raj
- Open Source Technology Center


When CONFIG_HOTPLUG_CPU is turned on we always use physflat mode (bigsmp) even 
when #of CPUs are less than 8 to avoid sending IPI to offline processors.

Without having BIGSMP on it spits out a warning during boot on systems that
seems misleading, since it complains even on systems that have less
than 8 cpus.

Signed-off-by: Ashok Raj <ashok.raj@intel.com>
---------------------------------------------------------

 arch/i386/Kconfig |    2 +-
 1 files changed, 1 insertion(+), 1 deletion(-)

Index: linux-2.6.16-rc6-mm1/arch/i386/Kconfig
===================================================================
--- linux-2.6.16-rc6-mm1.orig/arch/i386/Kconfig
+++ linux-2.6.16-rc6-mm1/arch/i386/Kconfig
@@ -760,7 +760,7 @@ config PHYSICAL_START
 
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
-	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER
+	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER && (X86_GENERICARCH || X86_BIGSMP)
 	---help---
 	  Say Y here to experiment with turning CPUs off and on.  CPUs
 	  can be controlled through /sys/devices/system/cpu.

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Andrew Morton]

Ashok Raj <ashok.raj@intel.com> wrote:
>
> When CONFIG_HOTPLUG_CPU is turned on we always use physflat mode (bigsmp) even 
>  when #of CPUs are less than 8 to avoid sending IPI to offline processors.
> 
>  Without having BIGSMP on it spits out a warning during boot on systems that
>  seems misleading, since it complains even on systems that have less
>  than 8 cpus.
> 
> ...
>
>  --- linux-2.6.16-rc6-mm1.orig/arch/i386/Kconfig
>  +++ linux-2.6.16-rc6-mm1/arch/i386/Kconfig
>  @@ -760,7 +760,7 @@ config PHYSICAL_START
>   
>   config HOTPLUG_CPU
>   	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
>  -	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER
>  +	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER && (X86_GENERICARCH || X86_BIGSMP)
>   	---help---
>   	  Say Y here to experiment with turning CPUs off and on.  CPUs
>   	  can be controlled through /sys/devices/system/cpu.

One of the main reasons for turning on CONFIG_HOTPLUG_CPU on x86 is
actually for suspend-to-disk on SMP.  I don't think it's desirable to force
all those little machines to use X86_GENERICARCH || X86_BIGSMP.  And it'd
be good to make that warning go away for 2.6.16.

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Ashok Raj]

On Mon, Mar 13, 2006 at 11:51:55AM -0800, Andrew Morton wrote:
> 
> One of the main reasons for turning on CONFIG_HOTPLUG_CPU on x86 is
> actually for suspend-to-disk on SMP.  I don't think it's desirable to force
> all those little machines to use X86_GENERICARCH || X86_BIGSMP.  And it'd
> be good to make that warning go away for 2.6.16.

But we cant use X86_PC since it uses logical flat mode for IPI's that could 
cause hangup's if we deliver IPI's using IPI broadcast shortcut.

In i386 we do have an alternate that would use mask value to deliver IPI's
but Andi's recommendataion was to use flat physical mode just like what we
do for X86_64.

Other than the IPI mode, are there any other things that hurt small systems
by choosing bigsmp mode?

Venki suggested we could make it !X86_PC instead of listing 
GENERICARCH or BIGSMP separately.


-- 
Cheers,
Ashok Raj
- Open Source Technology Center


When CONFIG_HOTPLUG_CPU is turned on we always use physflat mode (bigsmp) even 
when #of CPUs are less than 8 to avoid sending IPI to offline processors.

Without having BIGSMP on it spits out a warning during boot on systems that
seems misleading, since it complains even on systems that have less
than 8 cpus.

Signed-off-by: Ashok Raj <ashok.raj@intel.com>
---------------------------------------------------------

 arch/i386/Kconfig |    2 +-
 1 files changed, 1 insertion(+), 1 deletion(-)

Index: linux-2.6.16-rc6-mm1/arch/i386/Kconfig
===================================================================
--- linux-2.6.16-rc6-mm1.orig/arch/i386/Kconfig
+++ linux-2.6.16-rc6-mm1/arch/i386/Kconfig
@@ -760,7 +760,7 @@ config PHYSICAL_START
 
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
-	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER
+	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER && !X86_PC
 	---help---
 	  Say Y here to experiment with turning CPUs off and on.  CPUs
 	  can be controlled through /sys/devices/system/cpu.

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Andrew Morton]

Ashok Raj <ashok.raj@intel.com> wrote:
>
> 
> 
>  When CONFIG_HOTPLUG_CPU is turned on we always use physflat mode (bigsmp) even 
>  when #of CPUs are less than 8 to avoid sending IPI to offline processors.
> 
>  Without having BIGSMP on it spits out a warning during boot on systems that
>  seems misleading, since it complains even on systems that have less
>  than 8 cpus.
> 
>  Signed-off-by: Ashok Raj <ashok.raj@intel.com>
>  ---------------------------------------------------------
> 
>   arch/i386/Kconfig |    2 +-
>   1 files changed, 1 insertion(+), 1 deletion(-)
> 
>  Index: linux-2.6.16-rc6-mm1/arch/i386/Kconfig
>  ===================================================================
>  --- linux-2.6.16-rc6-mm1.orig/arch/i386/Kconfig
>  +++ linux-2.6.16-rc6-mm1/arch/i386/Kconfig
>  @@ -760,7 +760,7 @@ config PHYSICAL_START
>   
>   config HOTPLUG_CPU
>   	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
>  -	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER
>  +	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER && !X86_PC
>   	---help---
>   	  Say Y here to experiment with turning CPUs off and on.  CPUs
>   	  can be controlled through /sys/devices/system/cpu.

Still seems wrong.  People _do_ use HOTPLUG_CPU on X86_PCs so they can get
software suspend.  The number of people who do this are probably 100000x
the number of people who have physically hotpluggable CPUs.  And I don't
think we can churn their config requirements this much so late in the game.

So for now I suggest we're best off simply killing the printk (or doing
something smarter, like comparing cpu_online-map with cpu_possible_map
(which isn't right)).

Longer term, it appears that we need to do some Kconfig and C work to
separate out the HOTPLUG_CPU infrastructure which swsusp needs from actual
CPU hotplugging.

What _is_ this IPI problem anyway?  Can't send point-to-point IPIs to
offlined CPUs?  (Don't do that then?) Or do broadcast IPIs go wrong, or
what?

And does it affect pretend-x86-hotplug, or is it only affecting real hotplug?

Thanks.

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Ashok Raj]

On Mon, Mar 13, 2006 at 02:22:23PM -0800, Andrew Morton wrote:
> >   config HOTPLUG_CPU
> >   	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
> >  -	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER
> >  +	depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER && !X86_PC
> >   	---help---
> >   	  Say Y here to experiment with turning CPUs off and on.  CPUs
> >   	  can be controlled through /sys/devices/system/cpu.
> 
> Longer term, it appears that we need to do some Kconfig and C work to
> separate out the HOTPLUG_CPU infrastructure which swsusp needs from actual
> CPU hotplugging.

The needs are not any different. Both (swsusp and cpu hotplug) both require
logical cpu offlining which is what CONFIG_HOTPLUG_CPU does.

Physical cpu hotplug is enabled by CONFIG_ACPI_HOTPLUG_CPU.

> 
> What _is_ this IPI problem anyway?  Can't send point-to-point IPIs to
> offlined CPUs?  (Don't do that then?) Or do broadcast IPIs go wrong, or
> what?

Its not the point-to-point..we do that only to wake a CPU, but thats done
in flat physical mode always.

When we do smp_call_function() under X86_PC we use logical flat mode. 
This sends a broadcast IPI by using a shortcut message. This is bad, since 
the offline cpu may also receive it and process just when we bring the cpu 
online. 

send_IPI_allbutself() and send_IPI_all() versions that use the shortcut
values are the ones to avoid. 

> 
> And does it affect pretend-x86-hotplug, or is it only affecting real hotplug?
> 
its no more pretend-x86, in the past we used to put the cpu in idle(), 
now we do put the cpu in halt and bring back by another startup ipi, just like 
boot sequence, both for x86 and x86_64.

-- 
Cheers,
Ashok Raj
- Open Source Technology Center

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Nathan Lynch]

Ashok Raj wrote:
> On Mon, Mar 13, 2006 at 02:22:23PM -0800, Andrew Morton wrote:
> > 
> > And does it affect pretend-x86-hotplug, or is it only affecting real hotplug?
> > 
> its no more pretend-x86, in the past we used to put the cpu in idle(), 
> now we do put the cpu in halt and bring back by another startup ipi, just like 
> boot sequence, both for x86 and x86_64.

That's actually broken since 2.6.14 (at least on my P3 box); please
see:

Subject: i386 cpu hotplug bug - instant reboot when onlining secondary

http://lkml.org/lkml/2006/2/19/186

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Shaohua Li]

On Wed, 2006-03-15 at 13:44 +0800, Nathan Lynch wrote:
> Ashok Raj wrote: 
> > On Mon, Mar 13, 2006 at 02:22:23PM -0800, Andrew Morton wrote: 
> > >  
> > > And does it affect pretend-x86-hotplug, or is it only affecting
> real hotplug? 
> > >  
> > its no more pretend-x86, in the past we used to put the cpu in
> idle(),  
> > now we do put the cpu in halt and bring back by another startup ipi,
> just like  
> > boot sequence, both for x86 and x86_64.
> 
> That's actually broken since 2.6.14 (at least on my P3 box); please 
> see:
> 
> Subject: i386 cpu hotplug bug - instant reboot when onlining secondary
> 
> http://lkml.org/lkml/2006/2/19/186
Works for me. But I saw a warning.

---

 linux-2.6.15-root/arch/i386/kernel/cpu/common.c |    2 +-
 1 files changed, 1 insertion(+), 1 deletion(-)

diff -puN arch/i386/kernel/cpu/common.c~cpuhp arch/i386/kernel/cpu/common.c
--- linux-2.6.15/arch/i386/kernel/cpu/common.c~cpuhp	2006-03-14 12:13:43.000000000 +0800
+++ linux-2.6.15-root/arch/i386/kernel/cpu/common.c	2006-03-14 12:14:12.000000000 +0800
@@ -605,7 +605,7 @@ void __devinit cpu_init(void)
 		/* alloc_bootmem_pages panics on failure, so no check */
 		memset(gdt, 0, PAGE_SIZE);
 	} else {
-		gdt = (struct desc_struct *)get_zeroed_page(GFP_KERNEL);
+		gdt = (struct desc_struct *)get_zeroed_page(GFP_ATOMIC);
 		if (unlikely(!gdt)) {
 			printk(KERN_CRIT "CPU%d failed to allocate GDT\n", cpu);
 			for (;;)
_

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Andrew Morton]

Shaohua Li <shaohua.li@intel.com> wrote:
>
> On Wed, 2006-03-15 at 13:44 +0800, Nathan Lynch wrote:
>  > Ashok Raj wrote: 
>  > > On Mon, Mar 13, 2006 at 02:22:23PM -0800, Andrew Morton wrote: 
>  > > >  
>  > > > And does it affect pretend-x86-hotplug, or is it only affecting
>  > real hotplug? 
>  > > >  
>  > > its no more pretend-x86, in the past we used to put the cpu in
>  > idle(),  
>  > > now we do put the cpu in halt and bring back by another startup ipi,
>  > just like  
>  > > boot sequence, both for x86 and x86_64.
>  > 
>  > That's actually broken since 2.6.14 (at least on my P3 box); please 
>  > see:
>  > 
>  > Subject: i386 cpu hotplug bug - instant reboot when onlining secondary
>  > 
>  > http://lkml.org/lkml/2006/2/19/186
>  Works for me. But I saw a warning.

Guys, will you please stop being so cryptic?  What worked for you?  What
warning?  wtf is going on?  Who owns this problem, whatever it is?
<head spins>

>   linux-2.6.15-root/arch/i386/kernel/cpu/common.c |    2 +-
>   1 files changed, 1 insertion(+), 1 deletion(-)
> 
>  diff -puN arch/i386/kernel/cpu/common.c~cpuhp arch/i386/kernel/cpu/common.c
>  --- linux-2.6.15/arch/i386/kernel/cpu/common.c~cpuhp	2006-03-14 12:13:43.000000000 +0800
>  +++ linux-2.6.15-root/arch/i386/kernel/cpu/common.c	2006-03-14 12:14:12.000000000 +0800
>  @@ -605,7 +605,7 @@ void __devinit cpu_init(void)
>   		/* alloc_bootmem_pages panics on failure, so no check */
>   		memset(gdt, 0, PAGE_SIZE);
>   	} else {
>  -		gdt = (struct desc_struct *)get_zeroed_page(GFP_KERNEL);
>  +		gdt = (struct desc_struct *)get_zeroed_page(GFP_ATOMIC);

That would be rather a sad thing to have to do.  OK if it's during initial
bootup, less OK if it's during CPU hot-add.

[PATCH] No need to protect current->group_info in sys_getgroups(), in_group_p() and in_egroup_p()

[Eric Dumazet]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]


While doing some benchmarks of an Apache/PHP SMP server, I noticed high 
oprofile numbers in in_group_p() and _atomic_dec_and_lock().

rank  percent
  1     4.8911 % __link_path_walk
  2     4.8503 % __d_lookup
*3     4.2911 % _atomic_dec_and_lock
  4     3.9307 % __copy_to_user_ll
  5     4.9004 % sysenter_past_esp
*6     3.3248 % in_group_p

It appears that in_group_p() does an uncessary

get_group_info(current->group_info); /* atomic_inc() */
  ... /* access current->group_info */
put_group_info(current->group_info); /* _atomic_dec_and_lock */


It is not necessary to do this, because the current task holds a reference on 
its own group_info, and this reference cannot change during the lookup.

This patch deletes the get_group_info()/put_group_info() pair from 
sys_getgroups(), in_group_p() and in_egroup_p() functions.

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>


[-- Attachment #2: kernel_sys.patch --]
[-- Type: text/plain, Size: 884 bytes --]

--- a/kernel/sys.c	2006-03-15 10:14:37.000000000 +0100
+++ b/kernel/sys.c	2006-03-15 10:15:55.000000000 +0100
@@ -1433,7 +1433,6 @@
 		return -EINVAL;
 
 	/* no need to grab task_lock here; it cannot change */
-	get_group_info(current->group_info);
 	i = current->group_info->ngroups;
 	if (gidsetsize) {
 		if (i > gidsetsize) {
@@ -1446,7 +1445,6 @@
 		}
 	}
 out:
-	put_group_info(current->group_info);
 	return i;
 }
 
@@ -1487,9 +1485,7 @@
 {
 	int retval = 1;
 	if (grp != current->fsgid) {
-		get_group_info(current->group_info);
 		retval = groups_search(current->group_info, grp);
-		put_group_info(current->group_info);
 	}
 	return retval;
 }
@@ -1500,9 +1496,7 @@
 {
 	int retval = 1;
 	if (grp != current->egid) {
-		get_group_info(current->group_info);
 		retval = groups_search(current->group_info, grp);
-		put_group_info(current->group_info);
 	}
 	return retval;
 }

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Chuck Ebbert]

In-Reply-To: <Pine.LNX.4.64.0603120256480.14567@bizon.gios.gov.pl>

On Sun, 12 Mar 2006 03:04:40 +0100, Krzysztof Oledzki wrote:

> After upgrading to 2.6.16-rc6 I noticed this strange message:
> 
> More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
> Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
> 
> This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with 
> totoal of 4 logical CPUs).

In a later message, you wrote:

> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> Processor #0 15:4 APIC version 20
> ACPI: LAPIC (acpi_id[0x02] lapic_id[0x06] enabled)
> Processor #6 15:4 APIC version 20
             ^
> ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
> Processor #1 15:4 APIC version 20
> ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] enabled)
> Processor #7 15:4 APIC version 20
             ^

What processor numbers did you get on 2.6.15.x?
Does /proc/cpuinfo show all four CPUs?
If you start four CPU-hungry processes, do all four show 100% utilization in top(1)?


-- 
Chuck
"Penguins don't come from next door, they come from the Antarctic!"

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4637 bytes --]



On Wed, 15 Mar 2006, Chuck Ebbert wrote:

> In-Reply-To: <Pine.LNX.4.64.0603120256480.14567@bizon.gios.gov.pl>
>
> On Sun, 12 Mar 2006 03:04:40 +0100, Krzysztof Oledzki wrote:
>
>> After upgrading to 2.6.16-rc6 I noticed this strange message:
>>
>> More than 8 CPUs detected and CONFIG_X86_PC cannot handle it.
>> Use CONFIG_X86_GENERICARCH or CONFIG_X86_BIGSMP.
>>
>> This is a Dell PowerEdge SC1425 with two P4 Xeons with HT enabled (so with
>> totoal of 4 logical CPUs).
>
> In a later message, you wrote:
>
>> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
>> Processor #0 15:4 APIC version 20
>> ACPI: LAPIC (acpi_id[0x02] lapic_id[0x06] enabled)
>> Processor #6 15:4 APIC version 20
>             ^
>> ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
>> Processor #1 15:4 APIC version 20
>> ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] enabled)
>> Processor #7 15:4 APIC version 20
>             ^
>
> What processor numbers did you get on 2.6.15.x?

Mar  6 00:08:03 fw2 kernel: Processor #0 15:4 APIC version 20
Mar  6 00:08:03 fw2 kernel: Processor #6 15:4 APIC version 20
Mar  6 00:08:03 fw2 kernel: Processor #1 15:4 APIC version 20
Mar  6 00:08:03 fw2 kernel: Processor #7 15:4 APIC version 20


> Does /proc/cpuinfo show all four CPUs?
Yes.

# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 3
cpu MHz         : 3200.000
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr
bogomips        : 6404.87

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 3
cpu MHz         : 3200.000
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr
bogomips        : 6400.28

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 3
cpu MHz         : 2800.000
cache size      : 2048 KB
physical id     : 3
siblings        : 2
core id         : 0
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr
bogomips        : 6400.31

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 3
cpu MHz         : 2800.000
cache size      : 2048 KB
physical id     : 3
siblings        : 2
core id         : 0
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr
bogomips        : 6400.32


> If you start four CPU-hungry processes, do all four show 100% utilization in top(1)?

# cd /usr/src/linux
# /usr/bin/time make -j10 
(...)

Cpu0  : 94.4% us,  5.3% sy,  0.0% ni,  0.3% id,  0.0% wa,  0.0% hi,  0.0% si
Cpu1  : 92.4% us,  5.3% sy,  0.0% ni,  2.3% id,  0.0% wa,  0.0% hi,  0.0% si
Cpu2  : 91.7% us,  6.6% sy,  0.0% ni,  0.3% id,  1.3% wa,  0.0% hi,  0.0% si
Cpu3  : 94.7% us,  5.3% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si


Best regards,

 				Krzysztof Olędzki

Re: More than 8 CPUs detected and CONFIG_X86_PC cannot handle it on 2.6.16-rc6

[Ashok Raj]

On Tue, Mar 14, 2006 at 11:31:38PM -0800, Andrew Morton wrote:
> >  > see:
> >  > 
> >  > Subject: i386 cpu hotplug bug - instant reboot when onlining secondary
> >  > 
> >  > http://lkml.org/lkml/2006/2/19/186
> >  Works for me. But I saw a warning.
> 
> Guys, will you please stop being so cryptic?  What worked for you?  What
> warning?  wtf is going on?  Who owns this problem, whatever it is?

Nathan's problem is different, its nothing related to this thread.

Appears that a PIII box had trouble to bring a CPU back online after it was
just offlined. Iam not able to reproduce it with the systems i have here.
I have tried a PIII box itself, and also a x86_64 system booting a i386 kernel
and all seems to work ok.

Zwane was attempting to trace Nathan's issue with some experimental patches
but dont think it went far along yet.


> >  -		gdt = (struct desc_struct *)get_zeroed_page(GFP_KERNEL);
> >  +		gdt = (struct desc_struct *)get_zeroed_page(GFP_ATOMIC);
> 

It might help to post with the actual warning, so we can understand why this
fix is necessary.

-- 
Cheers,
Ashok Raj
- Open Source Technology Center

[PATCH] Use unsigned int types for a faster bsearch

[Eric Dumazet]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]

This patch avoids arithmetic on 'signed' types that are slower than 
'unsigned'. This saves space and cpu cycles.

size of kernel/sys.o before the patch (gcc-3.4.5)

    text    data     bss     dec     hex filename
   10924     252       4   11180    2bac kernel/sys.o

size of kernel/sys.o after the patch
    text    data     bss     dec     hex filename
   10903     252       4   11159    2b97 kernel/sys.o

I noticed that gcc-4.1.0 (from Fedora Core 5) even uses idiv instruction for 
(a+b)/2 if a and b are signed.

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>


[-- Attachment #2: groups_search.patch --]
[-- Type: text/plain, Size: 540 bytes --]

--- a/kernel/sys.c	2006-03-20 18:42:41.000000000 +0100
+++ b/kernel/sys.c	2006-03-20 19:00:43.000000000 +0100
@@ -1375,7 +1375,7 @@
 /* a simple bsearch */
 int groups_search(struct group_info *group_info, gid_t grp)
 {
-	int left, right;
+	unsigned int left, right;
 
 	if (!group_info)
 		return 0;
@@ -1383,7 +1383,7 @@
 	left = 0;
 	right = group_info->ngroups;
 	while (left < right) {
-		int mid = (left+right)/2;
+		unsigned int mid = (left+right)/2;
 		int cmp = grp - GROUP_AT(group_info, mid);
 		if (cmp > 0)
 			left = mid + 1;

[PATCH] Use __read_mostly on some hot fs variables

[Eric Dumazet]

[-- Attachment #1: Type: text/plain, Size: 773 bytes --]

I discovered on oprofile hunting on a SMP platform that dentry lookups were 
slowed down because d_hash_mask, d_hash_shift and dentry_hashtable were in a 
cache line that contained inodes_stat. So each time inodes_stats is changed by 
a cpu, other cpus have to refill their cache line.

This patch moves some variables to the __read_mostly section, in order to 
avoid false sharing. RCU dentry lookups can go full speed.

Before someone asks, it is valid to declare a pointer as 'read mostly', even 
if the data pointed by the pointer is heavily modified. hash table pointers 
and kmem_cache pointers are setup at boot time, so they are perfect candidates 
  to 'read_mostly' section. Same apply for 'struct vfsmount *'

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>



[-- Attachment #2: fs_readmostly.patch --]
[-- Type: text/plain, Size: 7330 bytes --]

--- a/fs/dcache.c	2006-03-21 13:48:13.000000000 +0100
+++ b/fs/dcache.c	2006-03-21 13:55:00.000000000 +0100
@@ -36,7 +36,7 @@
 
 /* #define DCACHE_DEBUG 1 */
 
-int sysctl_vfs_cache_pressure = 100;
+int sysctl_vfs_cache_pressure __read_mostly = 100;
 EXPORT_SYMBOL_GPL(sysctl_vfs_cache_pressure);
 
  __cacheline_aligned_in_smp DEFINE_SPINLOCK(dcache_lock);
@@ -44,7 +44,7 @@
 
 EXPORT_SYMBOL(dcache_lock);
 
-static kmem_cache_t *dentry_cache; 
+static kmem_cache_t *dentry_cache __read_mostly;
 
 #define DNAME_INLINE_LEN (sizeof(struct dentry)-offsetof(struct dentry,d_iname))
 
@@ -59,9 +59,9 @@
 #define D_HASHBITS     d_hash_shift
 #define D_HASHMASK     d_hash_mask
 
-static unsigned int d_hash_mask;
-static unsigned int d_hash_shift;
-static struct hlist_head *dentry_hashtable;
+static unsigned int d_hash_mask __read_mostly;
+static unsigned int d_hash_shift __read_mostly;
+static struct hlist_head *dentry_hashtable __read_mostly;
 static LIST_HEAD(dentry_unused);
 
 /* Statistics gathering. */
@@ -1706,10 +1706,10 @@
 }
 
 /* SLAB cache for __getname() consumers */
-kmem_cache_t *names_cachep;
+kmem_cache_t *names_cachep __read_mostly;
 
 /* SLAB cache for file structures */
-kmem_cache_t *filp_cachep;
+kmem_cache_t *filp_cachep __read_mostly;
 
 EXPORT_SYMBOL(d_genocide);
 
--- a/fs/inode.c	2006-03-21 13:50:19.000000000 +0100
+++ b/fs/inode.c	2006-03-21 13:54:39.000000000 +0100
@@ -56,8 +56,8 @@
 #define I_HASHBITS	i_hash_shift
 #define I_HASHMASK	i_hash_mask
 
-static unsigned int i_hash_mask;
-static unsigned int i_hash_shift;
+static unsigned int i_hash_mask __read_mostly;
+static unsigned int i_hash_shift __read_mostly;
 
 /*
  * Each inode can be on two separate lists. One is
@@ -73,7 +73,7 @@
 
 LIST_HEAD(inode_in_use);
 LIST_HEAD(inode_unused);
-static struct hlist_head *inode_hashtable;
+static struct hlist_head *inode_hashtable __read_mostly;
 
 /*
  * A simple spinlock to protect the list manipulations.
@@ -98,7 +98,7 @@
  */
 struct inodes_stat_t inodes_stat;
 
-static kmem_cache_t * inode_cachep;
+static kmem_cache_t * inode_cachep __read_mostly;
 
 static struct inode *alloc_inode(struct super_block *sb)
 {
--- a/fs/namespace.c	2006-03-22 05:20:33.000000000 +0100
+++ b/fs/namespace.c	2006-03-22 05:23:40.000000000 +0100
@@ -43,9 +43,9 @@
 
 static int event;
 
-static struct list_head *mount_hashtable;
+static struct list_head *mount_hashtable __read_mostly;
 static int hash_mask __read_mostly, hash_bits __read_mostly;
-static kmem_cache_t *mnt_cache;
+static kmem_cache_t *mnt_cache __read_mostly;
 static struct rw_semaphore namespace_sem;
 
 /* /sys/fs */
--- a/fs/dcookies.c	2006-03-22 05:35:46.000000000 +0100
+++ b/fs/dcookies.c	2006-03-22 05:36:55.000000000 +0100
@@ -37,9 +37,9 @@
 
 static LIST_HEAD(dcookie_users);
 static DECLARE_MUTEX(dcookie_sem);
-static kmem_cache_t * dcookie_cache;
-static struct list_head * dcookie_hashtable;
-static size_t hash_size;
+static kmem_cache_t * dcookie_cache __read_mostly;
+static struct list_head * dcookie_hashtable __read_mostly;
+static size_t hash_size __read_mostly;
 
 static inline int is_live(void)
 {
--- a/fs/fcntl.c	2006-03-22 05:37:40.000000000 +0100
+++ b/fs/fcntl.c	2006-03-22 05:43:49.000000000 +0100
@@ -413,7 +413,7 @@
 
 /* Table to convert sigio signal codes into poll band bitmaps */
 
-static long band_table[NSIGPOLL] = {
+static const long band_table[NSIGPOLL] = {
 	POLLIN | POLLRDNORM,			/* POLL_IN */
 	POLLOUT | POLLWRNORM | POLLWRBAND,	/* POLL_OUT */
 	POLLIN | POLLRDNORM | POLLMSG,		/* POLL_MSG */
@@ -532,7 +532,7 @@
 }
 
 static DEFINE_RWLOCK(fasync_lock);
-static kmem_cache_t *fasync_cache;
+static kmem_cache_t *fasync_cache __read_mostly;
 
 /*
  * fasync_helper() is used by some character device drivers (mainly mice)
--- a/fs/eventpoll.c	2006-03-22 05:39:06.000000000 +0100
+++ b/fs/eventpoll.c	2006-03-22 05:43:49.000000000 +0100
@@ -280,13 +280,13 @@
 static struct poll_safewake psw;
 
 /* Slab cache used to allocate "struct epitem" */
-static kmem_cache_t *epi_cache;
+static kmem_cache_t *epi_cache __read_mostly;
 
 /* Slab cache used to allocate "struct eppoll_entry" */
-static kmem_cache_t *pwq_cache;
+static kmem_cache_t *pwq_cache __read_mostly;
 
 /* Virtual fs used to allocate inodes for eventpoll files */
-static struct vfsmount *eventpoll_mnt;
+static struct vfsmount *eventpoll_mnt __read_mostly;
 
 /* File callbacks that implement the eventpoll file behaviour */
 static struct file_operations eventpoll_fops = {
--- a/fs/inotify.c	2006-03-22 05:40:44.000000000 +0100
+++ b/fs/inotify.c	2006-03-22 05:43:49.000000000 +0100
@@ -40,15 +40,15 @@
 static atomic_t inotify_cookie;
 static atomic_t inotify_watches;
 
-static kmem_cache_t *watch_cachep;
-static kmem_cache_t *event_cachep;
+static kmem_cache_t *watch_cachep __read_mostly;
+static kmem_cache_t *event_cachep __read_mostly;
 
-static struct vfsmount *inotify_mnt;
+static struct vfsmount *inotify_mnt __read_mostly;
 
 /* these are configurable via /proc/sys/fs/inotify/ */
-int inotify_max_user_instances;
-int inotify_max_user_watches;
-int inotify_max_queued_events;
+int inotify_max_user_instances __read_mostly;
+int inotify_max_user_watches __read_mostly;
+int inotify_max_queued_events __read_mostly;
 
 /*
  * Lock ordering:
--- a/fs/locks.c	2006-03-22 05:43:34.000000000 +0100
+++ b/fs/locks.c	2006-03-22 05:43:49.000000000 +0100
@@ -145,7 +145,7 @@
 
 static LIST_HEAD(blocked_list);
 
-static kmem_cache_t *filelock_cache;
+static kmem_cache_t *filelock_cache __read_mostly;
 
 /* Allocate an empty lock structure. */
 static struct file_lock *locks_alloc_lock(void)
--- a/fs/bio.c	2006-03-22 05:44:20.000000000 +0100
+++ b/fs/bio.c	2006-03-22 05:50:37.000000000 +0100
@@ -29,7 +29,7 @@
 
 #define BIO_POOL_SIZE 256
 
-static kmem_cache_t *bio_slab;
+static kmem_cache_t *bio_slab __read_mostly;
 
 #define BIOVEC_NR_POOLS 6
 
@@ -38,7 +38,7 @@
  * basically we just need to survive
  */
 #define BIO_SPLIT_ENTRIES 8	
-mempool_t *bio_split_pool;
+mempool_t *bio_split_pool __read_mostly;
 
 struct biovec_slab {
 	int nr_vecs;
--- a/fs/dnotify.c	2006-03-22 05:46:33.000000000 +0100
+++ b/fs/dnotify.c	2006-03-22 05:50:37.000000000 +0100
@@ -21,9 +21,9 @@
 #include <linux/spinlock.h>
 #include <linux/slab.h>
 
-int dir_notify_enable = 1;
+int dir_notify_enable __read_mostly = 1;
 
-static kmem_cache_t *dn_cache;
+static kmem_cache_t *dn_cache __read_mostly;
 
 static void redo_inode_mask(struct inode *inode)
 {
--- a/fs/block_dev.c	2006-03-22 05:48:29.000000000 +0100
+++ b/fs/block_dev.c	2006-03-22 05:50:37.000000000 +0100
@@ -238,7 +238,7 @@
  */
 
 static  __cacheline_aligned_in_smp DEFINE_SPINLOCK(bdev_lock);
-static kmem_cache_t * bdev_cachep;
+static kmem_cache_t * bdev_cachep __read_mostly;
 
 static struct inode *bdev_alloc_inode(struct super_block *sb)
 {
@@ -312,7 +312,7 @@
 	.kill_sb	= kill_anon_super,
 };
 
-static struct vfsmount *bd_mnt;
+static struct vfsmount *bd_mnt __read_mostly;
 struct super_block *blockdev_superblock;
 
 void __init bdev_cache_init(void)
--- a/fs/pipe.c	2006-03-22 05:51:16.000000000 +0100
+++ b/fs/pipe.c	2006-03-22 05:54:11.000000000 +0100
@@ -676,7 +676,7 @@
 	return NULL;
 }
 
-static struct vfsmount *pipe_mnt;
+static struct vfsmount *pipe_mnt __read_mostly;
 static int pipefs_delete_dentry(struct dentry *dentry)
 {
 	return 1;

Re: [PATCH] Use __read_mostly on some hot fs variables

[Nick Piggin]

Eric Dumazet wrote:

>
> Before someone asks, it is valid to declare a pointer as 'read 
> mostly', even if the data pointed by the pointer is heavily modified. 
> hash table pointers and kmem_cache pointers are setup at boot time, so 
> they are perfect candidates  to 'read_mostly' section. Same apply for 
> 'struct vfsmount *'
>

Yes... why wouldn't it be if the variable is only written to once? This
is _exactly_ what __read_mostly section is for, isn't it?

Patch looks good though.

Nick
---
Send instant messages to your online friends http://au.messenger.yahoo.com 

[RFC, PATCH] avoid some atomics in open()/close() for monothreaded processes

[Eric Dumazet]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

Goal : Avoid some locking/unlocking 'struct files_struct'->file_lock for mono 
threaded processes.

We define files_multithreaded() function .

static inline int files_multithreaded(const struct files_struct *files)
{
        return sizeof(files->file_lock) > 0 && atomic_read(&files->count) > 1;
}

On plain UP kernel (not preemptable nor spinlock debug), this function is a 
const 0, so that gcc can wipe out some code.

On preemptible or SMP, or spinlock debug kernels, the result is true only if 
the ref count is greater than 1 (multi threaded process or /proc/{pid}/fd is 
under investigation by another task)

This patch increases kernel size but pros are worth the cons, as said Linus 
himself, we should increase performance of mono-threaded tasks....

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>

[-- Attachment #2: files_multithreaded.patch --]
[-- Type: text/plain, Size: 1790 bytes --]

--- a/include/linux/file.h	2006-03-22 06:23:02.000000000 +0100
+++ b/include/linux/file.h	2006-03-22 07:11:08.000000000 +0100
@@ -42,6 +42,11 @@
 	spinlock_t file_lock;     /* Protects concurrent writers.  Nests inside tsk->alloc_lock */
 };
 
+static inline int files_multithreaded(const struct files_struct *files)
+{
+	return sizeof(files->file_lock) > 0 && atomic_read(&files->count) > 1;
+}
+
 #define files_fdtable(files) (rcu_dereference((files)->fdt))
 
 extern void FASTCALL(__fput(struct file *));
--- a/fs/open.c.orig	2006-03-22 06:24:34.000000000 +0100
+++ b/fs/open.c	2006-03-22 06:30:54.000000000 +0100
@@ -1050,11 +1050,17 @@
 {
 	struct files_struct *files = current->files;
 	struct fdtable *fdt;
-	spin_lock(&files->file_lock);
+	int fl_locked = 0;
+
+	if (files_multithreaded(files)) {
+		spin_lock(&files->file_lock);
+		fl_locked = 1;
+	}
 	fdt = files_fdtable(files);
 	BUG_ON(fdt->fd[fd] != NULL);
 	rcu_assign_pointer(fdt->fd[fd], file);
-	spin_unlock(&files->file_lock);
+	if (fl_locked)
+		spin_unlock(&files->file_lock);
 }
 
 EXPORT_SYMBOL(fd_install);
@@ -1147,8 +1153,12 @@
 	struct file * filp;
 	struct files_struct *files = current->files;
 	struct fdtable *fdt;
+	int fl_locked = 0;
 
-	spin_lock(&files->file_lock);
+	if (files_multithreaded(files)) {
+		spin_lock(&files->file_lock);
+		fl_locked = 1;
+	}
 	fdt = files_fdtable(files);
 	if (fd >= fdt->max_fds)
 		goto out_unlock;
@@ -1158,11 +1168,13 @@
 	rcu_assign_pointer(fdt->fd[fd], NULL);
 	FD_CLR(fd, fdt->close_on_exec);
 	__put_unused_fd(files, fd);
-	spin_unlock(&files->file_lock);
+	if (fl_locked)
+		spin_unlock(&files->file_lock);
 	return filp_close(filp, files);
 
 out_unlock:
-	spin_unlock(&files->file_lock);
+	if (fl_locked)
+		spin_unlock(&files->file_lock);
 	return -EBADF;
 }
 

Re: [RFC, PATCH] avoid some atomics in open()/close() for monothreaded processes

[Andrew Morton]

Eric Dumazet <dada1@cosmosbay.com> wrote:
>
> Goal : Avoid some locking/unlocking 'struct files_struct'->file_lock for mono 
> threaded processes.
> 
> We define files_multithreaded() function .
> 
> static inline int files_multithreaded(const struct files_struct *files)
> {
>         return sizeof(files->file_lock) > 0 && atomic_read(&files->count) > 1;
> }

That's bascially sizeof(spinlock_t).  That's architecture dependent and
varies wildly according to the day of week.

It _might_ work in all situations - probably you checked that.  But I still
wouldn't do it because it might break in the future.  Let's be explicit and
stick the appropriate ifdefs in there.

I'd also consider renaming it to files_shared() - processes are
multithreaded, not data structures.

Once you're done with that we should change fget_light() and fput_light() to
use this helper.  Separate patch.

Re: [RFC, PATCH] avoid some atomics in open()/close() for monothreaded processes

[Eric Dumazet]

Andrew Morton a écrit :
> Eric Dumazet <dada1@cosmosbay.com> wrote:
>> Goal : Avoid some locking/unlocking 'struct files_struct'->file_lock for mono 
>> threaded processes.
>>
>> We define files_multithreaded() function .
>>
>> static inline int files_multithreaded(const struct files_struct *files)
>> {
>>         return sizeof(files->file_lock) > 0 && atomic_read(&files->count) > 1;
>> }
> 
> That's bascially sizeof(spinlock_t).  That's architecture dependent and
> varies wildly according to the day of week.

I used sizeof(files->file_lock) instead of sizeof(spinlock_t) because I found 
it more explicit , while not using ugly ifdefs.

> 
> It _might_ work in all situations - probably you checked that.  But I still
> wouldn't do it because it might break in the future.  Let's be explicit and
> stick the appropriate ifdefs in there.
> 
> I'd also consider renaming it to files_shared() - processes are
> multithreaded, not data structures.

Thanks for the feedback, I will redo the patch and test it on various 
platforms before resubmit (including performance data :) )

> 
> Once you're done with that we should change fget_light() and fput_light() to
> use this helper.  Separate patch.

Hum... this discussion is not relevant with fget_light() unless I mistaken.

Nowadays, this function doesnt take spinlock thanks to RCU

Eric

Re: [RFC, PATCH] avoid some atomics in open()/close() for monothreaded processes

[Andrew Morton]

Eric Dumazet <dada1@cosmosbay.com> wrote:
>
> > 
>  > Once you're done with that we should change fget_light() and fput_light() to
>  > use this helper.  Separate patch.
> 
>  Hum... this discussion is not relevant with fget_light() unless I mistaken.

Take a look.  fget_light() uses essentially the same test as you do, for
the same reason.  So it's appropriate that fget_light() use this new helper
rather than open-coding it.