"boot: mem=1024M" causes only one CPU is brought up in 2.6.17-mm4

"boot: mem=1024M" causes only one CPU is brought up in 2.6.17-mm4

[Raid Cheng]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

I installed latest mm kernel 2.6.17-mm4 on ppc64 with RHEL4-U3.
When i tried to limit the memory to 1024M by modifying yaboot.conf or
typing on the boot line, I found some error messages in dmesg log
and only one CPU is brought up by checking /proc/cpuinfo.

 But if i boot the kernel without such "mem=1024M" limitation, all 8 CPUs
 will be show in /proc/cpuinfo

  I have seen this issue both with mm1,mm2,mm3 and mm4.

  The error message printed by xmon is like (will repeat for cpu1 - cpu7):
 -------------------------------------------------------------------------------------------------------
    cpu 0x1: Vector: 300 (Data Access) at [c00000000ffbed10]
    pc: 00000000000104dc
    lr: 000000000000830c
    sp: c00000000ffbef90
   msr: 8000000000001000
   dar: c00000000ffbef88
   dsisr: a000000
   current = 0xc000000002683820
   paca    = 0xc00000000055f080
   pid   = 0, comm = swapper
  WARNING: exception is not recoverable, can't continue
  Unable to handle kernel paging request for data at address 0xc00000000ffbef88
  Faulting instruction address: 0x000104dc

[-- Attachment #2: dmesg.2.6.17-mm4.log --]
[-- Type: application/octet-stream, Size: 12603 bytes --]

Using pSeries machine description
Page orders: linear mapping = 24, virtual = 12, io = 12
Found initrd at 0xc000000002500000:0xc0000000025dc000
Partition configured for 8 cpus.
Starting Linux PPC64 #1 SMP Tue Jun 27 01:56:47 EDT 2006
-----------------------------------------------------
ppc64_pft_size                = 0x1b
ppc64_interrupt_controller    = 0x2
physicalMemorySize            = 0x40000000
ppc64_caches.dcache_line_size = 0x80
ppc64_caches.icache_line_size = 0x80
htab_address                  = 0x0000000000000000
htab_hash_mask                = 0xfffff
-----------------------------------------------------
Linux version 2.6.17-mm4 (root@wakashi) (gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 SMP Tue Jun 27 01:56:47 EDT 2006
[boot]0012 Setup Arch
Node 0 Memory: 0x0-0x40000000
Node 1 Memory:
EEH: PCI Enhanced I/O Error Handling Enabled
PPC64 nvram contains 7168 bytes
Using dedicated idle loop
On node 0 totalpages: 262144
  DMA zone: 262144 pages, LIFO batch:31
On node 1 totalpages: 0
[boot]0015 Setup Done
Built 2 zonelists.  Total pages: 262144
Kernel command line: ro console=hvc0 rhgb quiet root=LABEL=/  mem=1024M 
[boot]0020 XICS Init
xics: no ISA interrupt controller
[boot]0021 XICS Done
PID hash table entries: 4096 (order: 12, 32768 bytes)
time_init: decrementer frequency = 188.047000 MHz
time_init: processor frequency   = 1504.376000 MHz
Console: colour dummy device 80x25
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
freeing bootmem node 0
Memory: 1014324k/1048576k available (5360k kernel code, 34252k reserved, 2032k data, 569k bss, 264k init)
Calibrating delay loop... 375.80 BogoMIPS (lpj=751616)
Mount-cache hash table entries: 256
Unable to handle kernel paging request for data at address 0xc00000000ffbef88
Faulting instruction address: 0x000104dc
Processor 1 is stuck.
Unable to handle kernel paging request for data at address 0xc00000000ffbdf88
Faulting instruction address: 0x000104dc
Processor 2 is stuck.
Unable to handle kernel paging request for data at address 0xc00000000ffbcf88
Faulting instruction address: 0x000104dc
Processor 3 is stuck.
Unable to handle kernel paging request for data at address 0xc00000000ffbbf88
Faulting instruction address: 0x000104dc
Processor 4 is stuck.
Unable to handle kernel paging request for data at address 0xc00000000ffbaf88
Faulting instruction address: 0x000104dc
Processor 5 is stuck.
Unable to handle kernel paging request for data at address 0xc00000000ffb9f88
Faulting instruction address: 0x000104dc
Processor 6 is stuck.
Unable to handle kernel paging request for data at address 0xc00000000ffb8f88
Faulting instruction address: 0x000104dc
Processor 7 is stuck.
Brought up 1 CPUs
Node 0 CPUs: 0
Node 1 CPUs:
migration_cost=0
checking if image is initramfs... it is
Freeing initrd memory: 880k freed
NET: Registered protocol family 16
PCI: Probing PCI hardware
IOMMU table initialized, virtual merging enabled
mapping IO 3fe00200000 -> d000080000000000, size: 100000
mapping IO 3fe00700000 -> d000080000100000, size: 100000
PCI: Probing PCI hardware done
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 6, 262144 bytes)
TCP established hash table entries: 131072 (order: 9, 2097152 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
vio_bus_init: processing c00000003fffa138
vio_bus_init: processing c00000003fffa300
vio_bus_init: processing c00000003fffa448
vio_bus_init: processing c00000003fffa5b0
RTAS daemon started
RTAS: event: 23, Type: Platform Error, Severity: 2
audit: initializing netlink socket (disabled)
audit(1151383045.640:1): initialized
Total HugeTLB memory allocated, 0
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
vio_register_driver: driver hvc_console registering
HVSI: registered 0 devices
Generic RTC Driver v1.07
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
RAMDISK driver initialized: 16 RAM disks of 65536K size 1024 blocksize
loop: loaded (max 8 devices)
Intel(R) PRO/1000 Network Driver - version 7.0.38-k4
Copyright (c) 1999-2006 Intel Corporation.
PCI: Enabling device: (0001:c8:01.0), cmd 143
e1000: 0001:c8:01.0: e1000_probe: (PCI-X:133MHz:64-bit) 00:02:55:2f:ee:ba
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
PCI: Enabling device: (0001:c8:01.1), cmd 143
e1000: 0001:c8:01.1: e1000_probe: (PCI-X:133MHz:64-bit) 00:02:55:2f:ee:bb
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
pcnet32.c:v1.32 18.Mar.2006 tsbogend@alpha.franken.de
e100: Intel(R) PRO/100 Network Driver, 3.5.10-k2-NAPI
e100: Copyright(c) 1999-2005 Intel Corporation
drivers/net/ibmveth.c: ibmveth: IBM i/pSeries Virtual Ethernet Driver 1.03
vio_register_driver: driver ibmveth registering
netconsole: not configured, aborting
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
st: Version 20050830, fixed bufsize 32768, s/g segs 256
PCI: Enabling device: (0000:c8:01.2), cmd 142
ehci_hcd 0000:c8:01.2: EHCI Host Controller
ehci_hcd 0000:c8:01.2: new USB bus registered, assigned bus number 1
ehci_hcd 0000:c8:01.2: irq 133, io mem 0x400b8002000
ehci_hcd 0000:c8:01.2: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb1: new device found, idVendor=0000, idProduct=0000
usb usb1: new device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 2.6.17-mm4 ehci_hcd
usb usb1: SerialNumber: 0000:c8:01.2
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ohci_hcd: 2006 May 24 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
ohci_hcd 0000:c8:01.0: OHCI Host Controller
ohci_hcd 0000:c8:01.0: new USB bus registered, assigned bus number 2
ohci_hcd 0000:c8:01.0: irq 133, io mem 0x400b8001000
usb usb2: new device found, idVendor=0000, idProduct=0000
usb usb2: new device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: OHCI Host Controller
usb usb2: Manufacturer: Linux 2.6.17-mm4 ohci_hcd
usb usb2: SerialNumber: 0000:c8:01.0
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 3 ports detected
ohci_hcd 0000:c8:01.1: OHCI Host Controller
ohci_hcd 0000:c8:01.1: new USB bus registered, assigned bus number 3
ohci_hcd 0000:c8:01.1: irq 133, io mem 0x400b8000000
usb usb3: new device found, idVendor=0000, idProduct=0000
usb usb3: new device strings: Mfr=3, Product=2, SerialNumber=1
usb usb3: Product: OHCI Host Controller
usb usb3: Manufacturer: Linux 2.6.17-mm4 ohci_hcd
usb usb3: SerialNumber: 0000:c8:01.1
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
Initializing USB Mass Storage driver...
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
mice: PS/2 mouse device common for all mice
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
device-mapper: ioctl: 4.8.0-ioctl (2006-06-24) initialised: dm-devel@redhat.com
oprofile: using ppc64/power5 performance monitoring.
Netfilter messages via NETLINK v0.30.
IPv4 over IPv4 tunneling driver
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
Freeing unused kernel memory: 264k freed
ipr: IBM Power RAID SCSI Device Driver version: 2.1.3 (March 29, 2006)
ipr 0001:d0:01.0: Found IOA with IRQ: 167
ipr 0001:d0:01.0: Starting IOA initialization sequence.
ipr 0001:d0:01.0: Adapter firmware version: 020A005C
ipr 0001:d0:01.0: IOA initialized.
scsi0 : IBM 570B Storage Adapter
  Vendor: IBM   H0  Model: HUS103036FL3800   Rev: RPQF
  Type:   Direct-Access                      ANSI SCSI revision: 04
SCSI device sda: 71096640 512-byte hdwr sectors (36401 MB)
sda: Write Protect is off
sda: Mode Sense: d3 00 10 08
SCSI device sda: drive cache: write through w/ FUA
SCSI device sda: 71096640 512-byte hdwr sectors (36401 MB)
sda: Write Protect is off
sda: Mode Sense: d3 00 10 08
SCSI device sda: drive cache: write through w/ FUA
 sda: sda1 sda2 sda3 sda4
sd 0:0:3:0: Attached scsi disk sda
sd 0:0:3:0: Attached scsi generic sg0 type 0
  Vendor: IBM   H0  Model: HUS103036FL3800   Rev: RPQF
  Type:   Direct-Access                      ANSI SCSI revision: 04
SCSI device sdb: 71096640 512-byte hdwr sectors (36401 MB)
sdb: Write Protect is off
sdb: Mode Sense: d3 00 10 08
SCSI device sdb: drive cache: write through w/ FUA
SCSI device sdb: 71096640 512-byte hdwr sectors (36401 MB)
sdb: Write Protect is off
sdb: Mode Sense: d3 00 10 08
SCSI device sdb: drive cache: write through w/ FUA
 sdb: unknown partition table
sd 0:0:4:0: Attached scsi disk sdb
sd 0:0:4:0: Attached scsi generic sg1 type 0
  Vendor: IBM   H0  Model: HUS103036FL3800   Rev: RPQF
  Type:   Direct-Access                      ANSI SCSI revision: 04
SCSI device sdc: 71096640 512-byte hdwr sectors (36401 MB)
sdc: Write Protect is off
sdc: Mode Sense: d3 00 10 08
SCSI device sdc: drive cache: write through w/ FUA
SCSI device sdc: 71096640 512-byte hdwr sectors (36401 MB)
sdc: Write Protect is off
sdc: Mode Sense: d3 00 10 08
SCSI device sdc: drive cache: write through w/ FUA
 sdc: sdc1 sdc2 sdc3 sdc4
sd 0:0:5:0: Attached scsi disk sdc
sd 0:0:5:0: Attached scsi generic sg2 type 0
  Vendor: IBM   H0  Model: HUS103036FL3800   Rev: RPQF
  Type:   Direct-Access                      ANSI SCSI revision: 04
SCSI device sdd: 71096640 512-byte hdwr sectors (36401 MB)
sdd: Write Protect is off
sdd: Mode Sense: d3 00 10 08
SCSI device sdd: drive cache: write through w/ FUA
SCSI device sdd: 71096640 512-byte hdwr sectors (36401 MB)
sdd: Write Protect is off
sdd: Mode Sense: d3 00 10 08
SCSI device sdd: drive cache: write through w/ FUA
 sdd: sdd1 sdd2 sdd3 sdd4
sd 0:0:8:0: Attached scsi disk sdd
sd 0:0:8:0: Attached scsi generic sg3 type 0
  Vendor: IBM       Model: VSBPD4E1  U4SCSI  Rev: 4770
  Type:   Enclosure                          ANSI SCSI revision: 02
scsi 0:0:15:0: Attached scsi generic sg4 type 13
  Vendor: IBM   H0  Model: ST336753LC        Rev: C51C
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sde: 71096640 512-byte hdwr sectors (36401 MB)
sde: Write Protect is off
sde: Mode Sense: cb 00 10 08
SCSI device sde: drive cache: write through w/ FUA
SCSI device sde: 71096640 512-byte hdwr sectors (36401 MB)
sde: Write Protect is off
sde: Mode Sense: cb 00 10 08
SCSI device sde: drive cache: write through w/ FUA
 sde:
sd 0:1:5:0: Attached scsi disk sde
sd 0:1:5:0: Attached scsi generic sg5 type 0
  Vendor: IBM   H0  Model: HUS103014FL3800   Rev: RPQF
  Type:   Direct-Access                      ANSI SCSI revision: 04
SCSI device sdf: 286748000 512-byte hdwr sectors (146815 MB)
sdf: Write Protect is off
sdf: Mode Sense: d3 00 10 08
SCSI device sdf: drive cache: write through w/ FUA
SCSI device sdf: 286748000 512-byte hdwr sectors (146815 MB)
sdf: Write Protect is off
sdf: Mode Sense: d3 00 10 08
SCSI device sdf: drive cache: write through w/ FUA
 sdf:
sd 0:1:8:0: Attached scsi disk sdf
sd 0:1:8:0: Attached scsi generic sg6 type 0
  Vendor: IBM       Model: VSBPD4E1  U4SCSI  Rev: 4770
  Type:   Enclosure                          ANSI SCSI revision: 02
scsi 0:1:15:0: Attached scsi generic sg7 type 13
scsi: unknown device type 31
  Vendor: IBM       Model: 570B001           Rev: 0150
  Type:   Unknown                            ANSI SCSI revision: 00
scsi 0:255:255:255: Attached scsi generic sg8 type 31
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
Floppy drive(s): fd0 is 2.88M
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
EXT3 FS on sda4, internal journal
device-mapper: multipath: version 1.0.5 loaded
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
e1000: eth0: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex

Re: "boot: mem=1024M" causes only one CPU is brought up in 2.6.17-mm4

[Michael Ellerman]

[-- Attachment #1: Type: text/plain, Size: 852 bytes --]

On Fri, 2006-06-30 at 15:05 +0800, Raid Cheng wrote:
> I installed latest mm kernel 2.6.17-mm4 on ppc64 with RHEL4-U3.
> When i tried to limit the memory to 1024M by modifying yaboot.conf or
> typing on the boot line, I found some error messages in dmesg log
> and only one CPU is brought up by checking /proc/cpuinfo.
> 
>  But if i boot the kernel without such "mem=1024M" limitation, all 8 CPUs
>  will be show in /proc/cpuinfo

Oops, must be a bug in my "cleaned up" mem=x handling. Can you send me
your System.map, and a tar.gz of /proc/device-tree for a working kernel
and also the mem=1024M kernel?

cheers

-- 
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: "boot: mem=1024M" causes only one CPU is brought up in 2.6.17-mm4

[Michael Ellerman]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

On Fri, 2006-06-30 at 15:05 +0800, Raid Cheng wrote:
> I installed latest mm kernel 2.6.17-mm4 on ppc64 with RHEL4-U3.
> When i tried to limit the memory to 1024M by modifying yaboot.conf or
> typing on the boot line, I found some error messages in dmesg log
> and only one CPU is brought up by checking /proc/cpuinfo.

I've just posted a fix:
"[PATCH] Fix mem= handling when the memory limit is > RMO".

cheers

-- 
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 191 bytes --]