[PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

This is a respin of the original page_offset and 64-bit page
table changes with some minor changes and a bug fix:

1) Bob's ACKs for the first 6 patches added.

2) Don't use "UL" postfix for shift counts, there is code that
   wants those to be "int", particularly printfs in mm/mm_init.c
   that use the "%d" format specifier for SECTIONS_SHIFT.

3) Adjust VA_HOLE defines to match what the new 3-level page
   tables can actually provide.

Signed-off-by: David S. Miller <davem@davemloft.net>

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 09/30/2013 09:31 AM, David Miller wrote:
> This is a respin of the original page_offset and 64-bit page
> table changes with some minor changes and a bug fix:
>
> 1) Bob's ACKs for the first 6 patches added.
>
> 2) Don't use "UL" postfix for shift counts, there is code that
>     wants those to be "int", particularly printfs in mm/mm_init.c
>     that use the "%d" format specifier for SECTIONS_SHIFT.
>
> 3) Adjust VA_HOLE defines to match what the new 3-level page
>     tables can actually provide.
>
> Signed-off-by: David S. Miller <davem@davemloft.net>
> --
> To unsubscribe from this list: send the line "unsubscribe sparclinux" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi,
Tested on 3 configurations, here is the result. I dont see exit_mmap 
warning anynmore.

And I think we need to increase NR_CPUS.

Thanks,
-Guru


On a T5:

Allocated 64 Megs of memory at 0x40000000 for kernel
Loaded kernel version 3.12.0
Loading initial ramdisk (30134996 bytes at 0x380004000000 phys, 
0x40C00000 virt)...
/
PROMLIB: Sun IEEE Boot Prom 'OBP 4.35.0.a 2013/03/01 14:53'
PROMLIB: Root node compatible: sun4v
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 3.12.0-16mainline.sparc64 (root@xxx) (gcc version 4.4.7 
20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Mon Sep 30 14:31:28 EDT 2013
bootconsole [earlyprom0] enabled
ARCH: SUN4V
Ethernet address: 00:10:e0:35:15:f6
PAGE_OFFSET is 0xffff800000000000 (max_phys_bits = 47)
Kernel: Using 5 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /virtual-devices@100/console@1
PROM: Built device tree with 967023 bytes of memory.
MDESC: Size is 812080 bytes.
PLATFORM: banner-name [SPARC T5-8]
PLATFORM: name [sun4v-platform]
PLATFORM: hostid [863515f6]
PLATFORM: serial# [003515f6]
PLATFORM: stick-frequency [3b9aca00]
PLATFORM: mac-address [10e03515f6]
PLATFORM: watchdog-resolution [1000 ms]
PLATFORM: watchdog-max-timeout [31536000000 ms]
PLATFORM: max-cpus [1024]
Ignoring CPU 64 which is >= NR_CPUS (64)
Ignoring CPU 65 which is >= NR_CPUS (64)




On a T4-1:

PROMLIB: Sun IEEE Boot Prom 'OBP 4.34.1 2012/08/01 12:19'
PROMLIB: Root node compatible: sun4v
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 3.12.0-16mainline.sparc64 (root@xxx) (gcc version 4.4.7 
20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Mon Sep 30 14:31:28 EDT 2013
bootconsole [earlyprom0] enabled
ARCH: SUN4V
Ethernet address: 00:21:28:e8:e9:da
PAGE_OFFSET is 0xffff800000000000 (max_phys_bits = 47)
Kernel: Using 5 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /virtual-devices@100/console@1
PROM: Built device tree with 166778 bytes of memory.
MDESC: Size is 60336 bytes.
PLATFORM: banner-name [SPARC T4-1]
PLATFORM: name [ORCL,SPARC-T4-1]
PLATFORM: hostid [85e8e9da]
PLATFORM: serial# [00e8e9da]
PLATFORM: stick-frequency [3b8f5850]
PLATFORM: mac-address [2128e8e9da]
PLATFORM: watchdog-resolution [1000 ms]
PLATFORM: watchdog-max-timeout [31536000000 ms]
PLATFORM: max-cpus [64]
Top of RAM: 0x7ffaf2000, Total RAM: 0x7df664000
Memory hole size: 516MB
Zone ranges:
   Normal   [mem 0x20400000-0x7ffaf1fff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x20400000-0x7ffa43fff]
   node   0: [mem 0x7ffa54000-0x7ffa67fff]
   node   0: [mem 0x7ffae6000-0x7ffaf1fff]
Booting Linux...
CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,vis3]
CPU CAPS: [hpc,ima,pause,cbcond,aes,des,kasumi,camellia]
CPU CAPS: [md5,sha1,sha256,sha512,mpmul,montmul,montsqr,crc32c]
PERCPU: Embedded 7 pages/cpu @ffff8007ef000000 s23296 r8192 d25856 u65536
SUN4V: Mondo queue sizes [cpu(8192) dev(16384) r(8192) nr(256)]
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 4095291
Policy zone: Normal
Kernel command line: root=/dev/mapper/VolGroup-lv_root ro rd_NO_LUKS 
LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap 
SYSFONT=latarcyrheb-sun16 rd_LVM_LV=VolGroup/lv_root KEYBOARDTYPE=pc 
KEYTABLE=us rd_NO_DM
PID hash table entries: 4096 (order: 2, 32768 bytes)
Sorting __ex_table...
Memory: 32735248K/33020304K available (5333K kernel code, 708K rwdata, 
2216K rodata, 488K init, 7896K bss, 285056K reserved)
Hierarchical RCU implementation.
NR_IRQS:255
clocksource: mult[801898] shift[23]
clockevent: mult[ffced917] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled, bootconsole disabled
PROMLIB: Sun IEEE Boot Prom 'OBP 4.34.1 2012/08/01 12:19'
PROMLIB: Root node compatible: sun4v
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 3.12.0-16mainline.sparc64 (root@xx) (gcc version 4.4.7 
20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Mon Sep 30 14:31:28 EDT 2013
bootconsole [earlyprom0] enabled
ARCH: SUN4V
Ethernet address: 00:21:28:e8:e9:da
PAGE_OFFSET is 0xffff800000000000 (max_phys_bits = 47)
Kernel: Using 5 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /virtual-devices@100/console@1
PROM: Built device tree with 166778 bytes of memory.
MDESC: Size is 60336 bytes.
PLATFORM: banner-name [SPARC T4-1]
PLATFORM: name [ORCL,SPARC-T4-1]
PLATFORM: hostid [85e8e9da]
PLATFORM: serial# [00e8e9da]
PLATFORM: stick-frequency [3b8f5850]
PLATFORM: mac-address [2128e8e9da]
PLATFORM: watchdog-resolution [1000 ms]
PLATFORM: watchdog-max-timeout [31536000000 ms]
PLATFORM: max-cpus [64]
Top of RAM: 0x7ffaf2000, Total RAM: 0x7df664000
Memory hole size: 516MB
Zone ranges:
   Normal   [mem 0x20400000-0x7ffaf1fff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x20400000-0x7ffa43fff]
   node   0: [mem 0x7ffa54000-0x7ffa67fff]
   node   0: [mem 0x7ffae6000-0x7ffaf1fff]
Booting Linux...
CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,vis3]
CPU CAPS: [hpc,ima,pause,cbcond,aes,des,kasumi,camellia]
CPU CAPS: [md5,sha1,sha256,sha512,mpmul,montmul,montsqr,crc32c]
PERCPU: Embedded 7 pages/cpu @ffff8007ef000000 s23296 r8192 d25856 u65536
SUN4V: Mondo queue sizes [cpu(8192) dev(16384) r(8192) nr(256)]
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 4095291
Policy zone: Normal
Kernel command line: root=/dev/mapper/VolGroup-lv_root ro rd_NO_LUKS 
LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap 
SYSFONT=latarcyrheb-sun16 rd_LVM_LV=VolGroup/lv_root KEYBOARDTYPE=pc 
KEYTABLE=us rd_NO_DM
PID hash table entries: 4096 (order: 2, 32768 bytes)
Sorting __ex_table...
Memory: 32735248K/33020304K available (5333K kernel code, 708K rwdata, 
2216K rodata, 488K init, 7896K bss, 285056K reserved)
Hierarchical RCU implementation.
NR_IRQS:255
clocksource: mult[801898] shift[23]
clockevent: mult[ffced917] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled, bootconsole disabled
allocated 67108864 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Calibrating delay using timer specific routine.. 2000.29 BogoMIPS 
(lpj@00592)
pid_max: default: 65536 minimum: 512
Security Framework initialized
SELinux:  Initializing.
Dentry cache hash table entries: 4194304 (order: 12, 33554432 bytes)
Inode-cache hash table entries: 2097152 (order: 11, 16777216 bytes)
Mount-cache hash table entries: 512
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
Initializing cgroup subsys hugetlb
ftrace: allocating 19188 entries in 38 pages
Performance events: Supported PMU type is 'niagara4'
Brought up 64 CPUs
Testing NMI watchdog ... OK.
devtmpfs: initialized
ldc.c:v1.1 (July 22, 2008)
regulator-dummy: no parameters
NET: Registered protocol family 16
VIO: Adding device channel-devices
VIO: Adding device vldc-port-0-0
VIO: Adding device vldc-port-0-1
VIO: Adding device vldc-port-0-2
VIO: Adding device vldc-port-1-0
VIO: Adding device vldc-port-2-0
VIO: Adding device vldc-port-3-0
VIO: Adding device vldc-port-3-1
VIO: Adding device vldc-port-3-2
VIO: Adding device vldc-port-3-3
VIO: Adding device vldc-port-3-4
VIO: Adding device vldc-port-3-6
VIO: Adding device vlds-port-0-0
VIO: Adding device ds-0
VIO: Adding device ds-1
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out



On a t4-1 LDOM:

Allocated 64 Megs of memory at 0x40000000 for kernel
Loaded kernel version 3.12.0
Loading initial ramdisk (31813824 bytes at 0x54800000 phys, 0x40C00000 
virt)...
|
PROMLIB: Sun IEEE Boot Prom 'OBP 4.34.3.a 2013/07/29 13:21'
PROMLIB: Root node compatible: sun4v
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 3.12.0-16mainline.sparc64 (root@xxxx) (gcc version 4.4.7 
20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Mon Sep 30 14:31:28 EDT 2013
bootconsole [earlyprom0] enabled
ARCH: SUN4V
Ethernet address: 00:14:4f:fb:36:6f
PAGE_OFFSET is 0xffff800000000000 (max_phys_bits = 47)
Kernel: Using 5 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /virtual-devices@100/console@1
PROM: Built device tree with 25094 bytes of memory.
MDESC: Size is 9680 bytes.
PLATFORM: banner-name [SPARC T4-1]
PLATFORM: name [ORCL,SPARC-T4-1]
PLATFORM: hostid [84fb366f]
PLATFORM: serial# [000cf710]
PLATFORM: stick-frequency [3b8f5850]
PLATFORM: mac-address [144ffb366f]
PLATFORM: watchdog-resolution [1000 ms]
PLATFORM: watchdog-max-timeout [31536000000 ms]
PLATFORM: max-cpus [64]
Top of RAM: 0x14fe54000, Total RAM: 0xff744000
Memory hole size: 1287MB
Zone ranges:
   Normal   [mem 0x50400000-0x14fe53fff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x50400000-0x14fb3bfff]
   node   0: [mem 0x14fe4c000-0x14fe53fff]
Booting Linux...
CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,vis3]
CPU CAPS: [hpc,ima,pause,cbcond,aes,des,kasumi,camellia]
CPU CAPS: [md5,sha1,sha256,sha512,mpmul,montmul,montsqr,crc32c]
PERCPU: Embedded 7 pages/cpu @ffff80014cc00000 s23296 r8192 d25856 u65536
SUN4V: Mondo queue sizes [cpu(8192) dev(16384) r(8192) nr(256)]
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 519079
Policy zone: Normal
Kernel command line: root=/dev/mapper/VolGroup-lv_root ro rd_NO_LUKS 
LANG=en_US.UTF-8 KEYTABLE=us rd_NO_MD rd_LVM_LV=VolGroup/lv_swap 
SYSFONT=latarcyrheb-sun16 rd_LVM_LV=VolGroup/lv_root rd_NO_DM
PID hash table entries: 4096 (order: 2, 32768 bytes)
Sorting __ex_table...
Memory: 4090816K/4185360K available (5333K kernel code, 708K rwdata, 
2216K rodata, 488K init, 7896K bss, 94544K reserved)
Hierarchical RCU implementation.
NR_IRQS:255
clocksource: mult[801898] shift[23]
clockevent: mult[ffced917] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled, bootconsole disabled
PROMLIB: Sun IEEE Boot Prom 'OBP 4.34.3.a 2013/07/29 13:21'
PROMLIB: Root node compatible: sun4v
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 3.12.0-16mainline.sparc64 (root@xxx) (gcc version 4.4.7 
20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Mon Sep 30 14:31:28 EDT 2013
bootconsole [earlyprom0] enabled
ARCH: SUN4V
Ethernet address: 00:14:4f:fb:36:6f
PAGE_OFFSET is 0xffff800000000000 (max_phys_bits = 47)
Kernel: Using 5 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /virtual-devices@100/console@1
PROM: Built device tree with 25094 bytes of memory.
MDESC: Size is 9680 bytes.
PLATFORM: banner-name [SPARC T4-1]
PLATFORM: name [ORCL,SPARC-T4-1]
PLATFORM: hostid [84fb366f]
PLATFORM: serial# [000cf710]
PLATFORM: stick-frequency [3b8f5850]
PLATFORM: mac-address [144ffb366f]
PLATFORM: watchdog-resolution [1000 ms]
PLATFORM: watchdog-max-timeout [31536000000 ms]
PLATFORM: max-cpus [64]
Top of RAM: 0x14fe54000, Total RAM: 0xff744000
Memory hole size: 1287MB
Zone ranges:
   Normal   [mem 0x50400000-0x14fe53fff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x50400000-0x14fb3bfff]
   node   0: [mem 0x14fe4c000-0x14fe53fff]
Booting Linux...
CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,vis3]
CPU CAPS: [hpc,ima,pause,cbcond,aes,des,kasumi,camellia]
CPU CAPS: [md5,sha1,sha256,sha512,mpmul,montmul,montsqr,crc32c]
PERCPU: Embedded 7 pages/cpu @ffff80014cc00000 s23296 r8192 d25856 u65536
SUN4V: Mondo queue sizes [cpu(8192) dev(16384) r(8192) nr(256)]
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 519079
Policy zone: Normal
Kernel command line: root=/dev/mapper/VolGroup-lv_root ro rd_NO_LUKS 
LANG=en_US.UTF-8 KEYTABLE=us rd_NO_MD rd_LVM_LV=VolGroup/lv_swap 
SYSFONT=latarcyrheb-sun16 rd_LVM_LV=VolGroup/lv_root rd_NO_DM
PID hash table entries: 4096 (order: 2, 32768 bytes)
Sorting __ex_table...
Memory: 4090816K/4185360K available (5333K kernel code, 708K rwdata, 
2216K rodata, 488K init, 7896K bss, 94544K reserved)
Hierarchical RCU implementation.
NR_IRQS:255
clocksource: mult[801898] shift[23]
clockevent: mult[ffced917] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled, bootconsole disabled
allocated 10485760 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Calibrating delay using timer specific routine.. 2014.90 BogoMIPS 
(lpj@29806)
pid_max: default: 65536 minimum: 512
Security Framework initialized
SELinux:  Initializing.
Dentry cache hash table entries: 524288 (order: 9, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 8, 2097152 bytes)
Mount-cache hash table entries: 512
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
Initializing cgroup subsys hugetlb
ftrace: allocating 19188 entries in 38 pages
Performance events: Supported PMU type is 'niagara4'
Brought up 2 CPUs
Testing NMI watchdog ...
WARNING: CPU#0: NMI appears to be stuck (0->0)!
Please report this to bugzilla.kernel.org,
and attach the output of the 'dmesg' command.

WARNING: CPU#1: NMI appears to be stuck (0->0)!
Please report this to bugzilla.kernel.org,
and attach the output of the 'dmesg' command.
devtmpfs: initialized
ldc.c:v1.1 (July 22, 2008)
regulator-dummy: no parameters
NET: Registered protocol family 16
VIO: Adding device channel-devices
VIO: Adding device vnet-port-0-0
VIO: Adding device vdc-port-0-0
VIO: Adding device vdc-port-1-0
VIO: Adding device vdc-port-2-0
VIO: Adding device vdc-port-3-0
VIO: Adding device vlds-port-0-0
VIO: Adding device ds-0
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Mon, 30 Sep 2013 13:14:01 -0700

> And I think we need to increase NR_CPUS.

Just set the config option to the appropriate value when you build, I
set it to 128 for my T4-2.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 09/30/2013 09:31 AM, David Miller wrote:
> This is a respin of the original page_offset and 64-bit page
> table changes with some minor changes and a bug fix:
>
> 1) Bob's ACKs for the first 6 patches added.
>
> 2) Don't use "UL" postfix for shift counts, there is code that
>     wants those to be "int", particularly printfs in mm/mm_init.c
>     that use the "%d" format specifier for SECTIONS_SHIFT.
>
> 3) Adjust VA_HOLE defines to match what the new 3-level page
>     tables can actually provide.
>
> Signed-off-by: David S. Miller <davem@davemloft.net>
> --
Allocate 10 hugepages, and run a test using tool 'fio'  with following 
jobfile.
Fio just hangs. Basically issue is with O_DIRECT and hugepages. If you 
comment
out "direct=1" from jobfile test works fine.


[global]
bs=8k
randrepeat=1
mem=shmhuge
size\x100m
direct=1
[job2]
ioengine=psync
rw=randrw
filename=file1:file2

cat  /proc/49667/stack
[<00000000004209d4>] tl0_irq14+0x14/0x20
[<0000000000570ffc>] hugetlb_fault+0xbc/0x360
[<000000000057135c>] follow_hugetlb_page+0xbc/0x300
[<000000000055526c>] __get_user_pages+0x2ec/0x3a0
[<00000000005553e4>] get_user_pages+0x44/0x60
[<0000000000457590>] get_user_pages_fast+0xf0/0x220
[<00000000005ca878>] do_direct_IO+0xed8/0x1c80
[<00000000005cbb44>] __blockdev_direct_IO+0x344/0x1640
[<0000000010320444>] ext4_ind_direct_IO+0xe4/0x440 [ext4]
[<00000000102dc888>] ext4_ext_direct_IO+0x1c8/0x2c0 [ext4]
[<00000000102dca7c>] ext4_direct_IO+0xfc/0x200 [ext4]
[<000000000052d4ac>] generic_file_aio_read+0x1ec/0x220
[<000000000058dd98>] do_sync_read+0x58/0xa0
[<000000000058f42c>] vfs_read+0x6c/0x140
[<000000000058f568>] SyS_pread64+0x68/0x80
[<00000000004061d4>] linux_sparc_syscall+0x34/0x44


[root@localhost ~]# cat  /proc/49667/stack
[<0000000000538f58>] put_compound_page+0x38/0x1c0
[<000002eab77ca457>] 0x2eab77ca457


[root@localhost ~]# cat  /proc/49667/stack
[<0000000000538f2c>] put_compound_page+0xc/0x1c0
[<000000000053912c>] put_page+0x4c/0x60
[<0000000000571124>] hugetlb_fault+0x1e4/0x360
[<000000000057135c>] follow_hugetlb_page+0xbc/0x300
[<000000000055526c>] __get_user_pages+0x2ec/0x3a0
[<00000000005553e4>] get_user_pages+0x44/0x60
[<0000000000457590>] get_user_pages_fast+0xf0/0x220
[<00000000005ca878>] do_direct_IO+0xed8/0x1c80
[<00000000005cbb44>] __blockdev_direct_IO+0x344/0x1640
[<0000000010320444>] ext4_ind_direct_IO+0xe4/0x440 [ext4]
[<00000000102dc888>] ext4_ext_direct_IO+0x1c8/0x2c0 [ext4]
[<00000000102dca7c>] ext4_direct_IO+0xfc/0x200 [ext4]
[<000000000052d4ac>] generic_file_aio_read+0x1ec/0x220
[<000000000058dd98>] do_sync_read+0x58/0xa0
[<000000000058f42c>] vfs_read+0x6c/0x140
[<000000000058f568>] SyS_pread64+0x68/0x80
[<00000000004061d4>] linux_sparc_syscall+0x34/0x44

thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Mon, 30 Sep 2013 16:47:48 -0700

> Allocate 10 hugepages, and run a test using tool 'fio' with following
> jobfile.
> Fio just hangs. Basically issue is with O_DIRECT and hugepages. If you
> comment
> out "direct=1" from jobfile test works fine.

This worked before my changes right?

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 09/30/2013 04:51 PM, David Miller wrote:
> From: Gurudas Pai <gurudas.pai@oracle.com>
> Date: Mon, 30 Sep 2013 16:47:48 -0700
>
>> Allocate 10 hugepages, and run a test using tool 'fio' with following
>> jobfile.
>> Fio just hangs. Basically issue is with O_DIRECT and hugepages. If you
>> comment
>> out "direct=1" from jobfile test works fine.
> This worked before my changes right?
No, Just now tried with old kernel, it hangs as well.

Thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Mon, 30 Sep 2013 17:42:02 -0700

> On 09/30/2013 04:51 PM, David Miller wrote:
>> From: Gurudas Pai <gurudas.pai@oracle.com>
>> Date: Mon, 30 Sep 2013 16:47:48 -0700
>>
>>> Allocate 10 hugepages, and run a test using tool 'fio' with following
>>> jobfile.
>>> Fio just hangs. Basically issue is with O_DIRECT and hugepages. If you
>>> comment
>>> out "direct=1" from jobfile test works fine.
>> This worked before my changes right?
> No, Just now tried with old kernel, it hangs as well.

I can reproduce this so I'll have a look, thanks.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 09/30/2013 09:31 AM, David Miller wrote:
> This is a respin of the original page_offset and 64-bit page
> table changes with some minor changes and a bug fix:
>
> 1) Bob's ACKs for the first 6 patches added.
>
> 2) Don't use "UL" postfix for shift counts, there is code that
>     wants those to be "int", particularly printfs in mm/mm_init.c
>     that use the "%d" format specifier for SECTIONS_SHIFT.
>
> 3) Adjust VA_HOLE defines to match what the new 3-level page
>     tables can actually provide.
>
> Signed-off-by: David S. Miller <davem@davemloft.net>
> --
Run a java load with hugepages and we see following error. (will try 
without these patches shortly)

BUG: soft lockup - CPU#28 stuck for 22s! [java:23678]
Modules linked in: ext4(F) jbd2(F) mbcache(F) dm_round_robin(F) 
scsi_dh_emc(F) usb_storage(F) sr_mod(F) cdrom(F) sd_mod(F) 
megaraid_sas(F) ixgbe(F) igb(F) hwmon(F) i2c_algo_bit(F) i2c_core(F) 
ptp(F) pps_core(F) lpfc(F) scsi_transport_fc(F) scsi_tgt(F) 
crc_t10dif(F) crct10dif_common(F) mpt2sas(F) scsi_transport_sas(F) 
raid_class(F) camellia_sparc64(F) sha256_sparc64(F) aes_sparc64(F) 
sha512_sparc64(F) crc32c_sparc64(F) des_sparc64(F) des_generic(F) 
dm_multipath(F) sunvnet(F) sunvdc(F) dm_mirror(F) dm_region_hash(F) 
dm_log(F) dm_mod(F) be2iscsi(F) bnx2i(F) cnic(F) uio(F) ipv6(F) 
cxgb4i(F) cxgb4(F) cxgb3i(F) libcxgbi(F) cxgb3(F) mdio(F) 
libiscsi_tcp(F) qla4xxx(F) iscsi_boot_sysfs(F) libiscsi(F) 
scsi_transport_iscsi(F)
CPU: 28 PID: 23678 Comm: java Tainted: GF 3.12.0-16mainline.sparc64 #1
task: ffff800fc71c28e0 ti: ffff800e48290000 task.ti: ffff800e48290000
TSTATE: 0000004411001604 TPC: 000000000043eb10 TNPC: 000000000043eb14 Y: 
00000000    Tainted: GF
TPC: <xcall_deliver+0x110/0x140>
g0: 00000000004209d0 g1: 0000000000000000 g2: 000000000000007f g3: 
0000000000000001
g4: ffff800fc71c28e0 g5: ffff800fde52c000 g6: ffff800e48290000 g7: 
0000000000000006
o0: 0000000000c9bd40 o1: 000000000000003f o2: 0000000000000040 o3: 
000000000000003e
o4: 000000000000007e o5: 0000000000002710 sp: ffff800e48292a21 ret_pc: 
000000000043eb20
RPC: <xcall_deliver+0x120/0x140>
l0: 000000000000003f l1: 0000000000b8290c l2: 000000000000001c l3: 
000000000093db10
l4: ffff8000207d8040 l5: 0000000000c9bd40 l6: 0000000000000000 l7: 
0000000000000008
i0: 0000038800456034 i1: fffffc0448050000 i2: 0000000000000000 i3: 
ffff800025a35470
i4: 0000000000000000 i5: 000050d808df29c7 i6: ffff800e48292ad1 i7: 
000000000043fca4
I7: <smp_flush_tlb_page+0x44/0xc0>
Call Trace:
  [000000000043fca4] smp_flush_tlb_page+0x44/0xc0
  [0000000000456758] tlb_batch_add+0x178/0x1c0
  [0000000000458ccc] huge_ptep_get_and_clear+0x8c/0xc0
  [0000000000570654] __unmap_hugepage_range+0x134/0x280
  [000000000057153c] __unmap_hugepage_range_final+0x1c/0x40
  [000000000055771c] unmap_single_vma+0x9c/0xc0
  [0000000000557768] unmap_vmas+0x28/0x60
  [000000000055a0c8] exit_mmap+0x88/0x160
  [0000000000465014] mmput+0x54/0xe0
  [000000000046841c] exit_mm+0x13c/0x1a0
  [0000000000469ab0] do_exit+0x130/0x440
  [0000000000469df0] do_group_exit+0x30/0xc0
  [000000000047ab70] get_signal_to_deliver+0x1f0/0x4c0
  [000000000042d674] do_signal+0x54/0x540
  [000000000042dba8] do_notify_resume+0x48/0x80
  [0000000000404b04] __handle_signal+0xc/0x2c



And in few minutes :

SUN4V-DTLB: Error at TPC[fffffc01009e9af4], tl 1
SUN4V-DTLB: TPC<0xfffffc01009e9af4>
SUN4V-DTLB: O7[fffffc01093839d0]
SUN4V-DTLB: O7<0xfffffc01093839d0>
SUN4V-DTLB: vaddr[fffffc01000bc000] ctx[1c6c] pte[9a000000000f0610] error[2]

Machine hard hung.

Thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 09/30/2013 04:07 PM, David Miller wrote:
> From: Gurudas Pai <gurudas.pai@oracle.com>
> Date: Mon, 30 Sep 2013 13:14:01 -0700
>
>> And I think we need to increase NR_CPUS.
> Just set the config option to the appropriate value when you build, I
> set it to 128 for my T4-2.
Sure, also it is running out of NR_IRQ. I agree this is not related to 
47-bit bit patches.

IRQ: Out of virtual IRQs.
IRQ: Out of virtual IRQs.
IRQ: Out of virtual IRQs.
IRQ: Out of virtual IRQs.

Thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Tue, 01 Oct 2013 16:08:08 -0700

> Run a java load with hugepages and we see following error. (will try
> without these patches shortly)

Regardless, you're going to have to reproduce this with a C or similar
test case, I'm not running java to debug something.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Mon, 30 Sep 2013 17:42:02 -0700

> On 09/30/2013 04:51 PM, David Miller wrote:
>> From: Gurudas Pai <gurudas.pai@oracle.com>
>> Date: Mon, 30 Sep 2013 16:47:48 -0700
>>
>>> Allocate 10 hugepages, and run a test using tool 'fio' with following
>>> jobfile.
>>> Fio just hangs. Basically issue is with O_DIRECT and hugepages. If you
>>> comment
>>> out "direct=1" from jobfile test works fine.
>> This worked before my changes right?
> No, Just now tried with old kernel, it hangs as well.

This should fix it:

diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h
index d06079c..99b490b 100644
--- a/include/asm-generic/hugetlb.h
+++ b/include/asm-generic/hugetlb.h
@@ -6,12 +6,12 @@ static inline pte_t mk_huge_pte(struct page *page, pgprot_t pgprot)
 	return mk_pte(page, pgprot);
 }
 
-static inline int huge_pte_write(pte_t pte)
+static inline unsigned long huge_pte_write(pte_t pte)
 {
 	return pte_write(pte);
 }
 
-static inline int huge_pte_dirty(pte_t pte)
+static inline unsigned long huge_pte_dirty(pte_t pte)
 {
 	return pte_dirty(pte);
 }

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Tue, 01 Oct 2013 16:08:08 -0700

> Run a java load with hugepages and we see following error. (will try
> without these patches shortly)

So does it trigger without the 47-bit patches applied?

Thanks.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

>> Run a java load with hugepages and we see following error. (will try
>> without these patches shortly)
> So does it trigger without the 47-bit patches applied?
Yes, it triggers without patches. Will try to re-produce with small 
testcase,
might take a while though.

Thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Tue, 01 Oct 2013 18:20:07 -0700

> On 09/30/2013 04:07 PM, David Miller wrote:
>> From: Gurudas Pai <gurudas.pai@oracle.com>
>> Date: Mon, 30 Sep 2013 13:14:01 -0700
>>
>>> And I think we need to increase NR_CPUS.
>> Just set the config option to the appropriate value when you build, I
>> set it to 128 for my T4-2.
> Sure, also it is running out of NR_IRQ. I agree this is not related to
> 47-bit bit patches.
> 
> IRQ: Out of virtual IRQs.
> IRQ: Out of virtual IRQs.
> IRQ: Out of virtual IRQs.
> IRQ: Out of virtual IRQs.

Can you send me the output of "/usr/sbin/prtconf -pv" and a machine
description dump from this machine?

You can get the raw binary machine description blob under linux into a
file by simply doing a large enough read() on /dev/mdesc.  The read()
system call will return -EMSGSIZE until you pass in a large enough
sized buffer.

Thanks.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

>>>> Allocate 10 hugepages, and run a test using tool 'fio' with following
>>>> jobfile.
>>>> Fio just hangs. Basically issue is with O_DIRECT and hugepages. If you
>>>> comment
>>>> out "direct=1" from jobfile test works fine.
>>> This worked before my changes right?
>> No, Just now tried with old kernel, it hangs as well.
> This should fix it:
>
> diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h
> index d06079c..99b490b 100644
> --- a/include/asm-generic/hugetlb.h
> +++ b/include/asm-generic/hugetlb.h
> @@ -6,12 +6,12 @@ static inline pte_t mk_huge_pte(struct page *page, pgprot_t pgprot)
>   	return mk_pte(page, pgprot);
>   }
>   
> -static inline int huge_pte_write(pte_t pte)
> +static inline unsigned long huge_pte_write(pte_t pte)
>   {
>   	return pte_write(pte);
>   }
>   
> -static inline int huge_pte_dirty(pte_t pte)
> +static inline unsigned long huge_pte_dirty(pte_t pte)
>   {
>   	return pte_dirty(pte);
>   }
Yep, issue resolved, thanks !

Thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Wed, 02 Oct 2013 14:02:18 -0700

> Yep, issue resolved, thanks !

Thanks for testing, I'll get this integrated upstream soon.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 09/30/2013 09:31 AM, David Miller wrote:
> This is a respin of the original page_offset and 64-bit page
> table changes with some minor changes and a bug fix:
>
> 1) Bob's ACKs for the first 6 patches added.
>
> 2) Don't use "UL" postfix for shift counts, there is code that
>     wants those to be "int", particularly printfs in mm/mm_init.c
>     that use the "%d" format specifier for SECTIONS_SHIFT.
>
> 3) Adjust VA_HOLE defines to match what the new 3-level page
>     tables can actually provide.
>
> Signed-off-by: David S. Miller <davem@davemloft.net>
> --
On a t4-1 machine with 64gb RAM , run following testcase as

./a.out 10000

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>

int main(int ac, char **av) {
     long int n,m,i;
     char *buf;

     if(ac < 2) {
         printf("Usage: %s <alloc_size_MB>\n",av[0]);
         exit(1);
     }
     sscanf(av[1],"%ld",&n);
     m=n*1024*1024;
     printf("Allocating %ld bytes (%d MB), please wait...\n",m,n);
     buf=malloc(n*1024*1024);
       for(i=0;i<m;*(buf+i++)=1)
                 ;
     printf("Sleeping ...\n");
     sleep(36000);
}


And run echo 1000000 > /proc/sys/vm/nr_hugepages couple of times. 
Machine becomes unusuable.
This includes "fix generic hugetlb pte check return type." patch as 
well.Oom killers and then lockups.
Will try wothout patches very soon.

Full message: http://pastebin.com/VW0Uxpqd



BUG: soft lockup - CPU#15 stuck for 22s! [abrt-action-sav:3155]
Modules linked in: ext4 jbd2 mbcache usb_storage sr_mod cdrom sd_mod 
megaraid_sas ixgbe igb hwmon i2c_algo_bit i2c_core ptp pps_core lpfc 
scsi_transport_fc scsi_tgt crc_t10dif crct10dif_common mpt2sas 
scsi_transport_sas raid_class des_sparc64 des_generic sha512_sparc64 
aes_sparc64 sha256_sparc64 crc32c_sparc64 camellia_sparc64 sunvnet 
sunvdc dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio 
ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx 
iscsi_boot_sysfs libiscsi scsi_transport_iscsi
CPU: 15 PID: 3155 Comm: abrt-action-sav Not tainted 
3.12.0-16mainline3.1.sparc64 #1
task: ffff800fcc66e620 ti: ffff800fcb46c000 task.ti: ffff800fcb46c000
TSTATE: 0000004411001606 TPC: 000000000052e7f0 TNPC: 000000000052e7f4 Y: 
00000000    Not tainted
TPC: <oom_kill_process+0x150/0x360>
g0: 0000000000b838e8 g1: ffff800fc5c06108 g2: 0000000000000000 g3: 
ffff800fde82ff8c
g4: ffff800fcc66e620 g5: ffff800fde45c000 g6: ffff800fcb46c000 g7: 
0000000000000720
o0: 0000000000b68000 o1: 0000000000ac56f0 o2: 0000000000000a7d o3: 
ffff800fc7f310a0
o4: 0000000000000000 o5: 0000000000000fd3 sp: ffff800fcb46ee51 ret_pc: 
000000000052e750
RPC: <oom_kill_process+0xb0/0x360>
l0: 0000000000000000 l1: 0000000000000000 l2: ffff800fc7f30c20 l3: 
ffff800fc5c05d80
l4: 0000000000b68000 l5: ffff800fff9dfad8 l6: 0000000000000034 l7: 
0000000001425e80
i0: ffff800fc7f30c20 i1: ffff800fc5c06078 i2: ffff800fc5c05d70 i3: 
ffff800fcb46f708
i4: 00000000007df02e i5: 0000000000000000 i6: ffff800fcb46ef11 i7: 
000000000052ec5c
I7: <out_of_memory+0x25c/0x2c0>
Call Trace:
  [000000000052ec5c] out_of_memory+0x25c/0x2c0
  [0000000000534830] __alloc_pages_nodemask+0x9d0/0xaa0
  [0000000000574e70] alloc_pages_vma+0x70/0x180
  [0000000000552cd4] __do_fault+0x294/0x520
  [00000000005543b8] handle_pte_fault+0xb8/0x240
  [0000000000554ca4] __handle_mm_fault+0xa4/0x140
  [0000000000554de4] handle_mm_fault+0xa4/0x180
  [0000000000936954] do_sparc64_fault+0x3f4/0x7c0
  [0000000000407a58] sparc64_realfault_common+0x10/0x20
BUG: NMI Watchdog detected LOCKUP on CPU19, ip 00934fc8, registers:
CPU: 19 PID: 3087 Comm: login Not tainted 3.12.0-16mainline3.1.sparc64 #1
task: ffff800fc8ffa120 ti: ffff800fcda10000 task.ti: ffff800fcda10000
TSTATE: 0000000011e01604 TPC: 0000000000934fc8 TNPC: 0000000000934fc0 Y: 
00000000    Not tainted
TPC: <_raw_spin_trylock_bh+0xc8/0x100>
g0: 0000000000040000 g1: 0000000080000000 g2: 0000000000000001 g3: 
0000000080000001
g4: ffff800fc8ffa120 g5: ffff800fde49c000 g6: ffff800fcda10000 g7: 
000000000000aaf2
o0: 00000000524e1b0c o1: 000000000501b316 o2: 0000000000000000 o3: 
0000000000000000
o4: 0000000000000000 o5: 0000000000000000 sp: ffff800fcda13541 ret_pc: 
00000000004e8064
RPC: <__audit_syscall_entry+0xa4/0x120>
l0: ffff800fcde31c00 l1: 0000000000000000 l2: 0000000000000000 l3: 
ffff800fcb022378
l4: 000000008c3439ca l5: 000007feff9a5298 l6: 000007feff9a5280 l7: 
fffffc01001254c8
i0: 0000000000b68000 i1: 0000000000000000 i2: 0000000000b9cbc8 i3: 
0000000000000000
i4: fffffffffffffff8 i5: 0000000000000000 i6: ffff800fcda135f1 i7: 
000000000047cd10
I7: <SyS_setpgid+0x30/0x180>
Call Trace:
  [000000000047cd10] SyS_setpgid+0x30/0x180
  [00000000004061d4] linux_sparc_syscall+0x34/0x44
CPU: 19 PID: 3087 Comm: login Not tainted 3.12.0-16mainline3.1.sparc64 #1
Call Trace:
  [000000000043dd10] die_nmi.clone.0+0x90/0x120
  [00000000009355a4] perfctr_irq+0x224/0x260
  [00000000004209f4] tl0_irq15+0x14/0x20
  [0000000000934fc8] _raw_spin_trylock_bh+0xc8/0x100
  [000000000047cd10] SyS_setpgid+0x30/0x180
  [00000000004061d4] linux_sparc_syscall+0x34/0x44

INFO: rcu_sched self-detected stall on CPU { 15} INFO: rcu_sched 
detected stalls on CPUs/tasks: { 15} (detected by 21, t\x15002 jiffies, 
g\x1155, c\x1154, q\x198)
   CPU[  0]: TSTATE[0000000080001605] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/0:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[  1]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/1:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[  2]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/2:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[  3]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/3:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[  4]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/4:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[  5]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/5:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[  6]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/6:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[  7]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/7:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[  8]: TSTATE[0000004480001605] TPC[00000000006a5390] 
TNPC[00000000006a5394] TASK[swapper/8:0]
              TPC[atomic_add+0x10/0x54] 
O7[rcu_eqs_enter_common+0x28/0xe0] I7[rcu_idle_enter+0x64/0xe0] 
RPC[cpu_idle_loop+0x134/0x280]
   CPU[  9]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/9:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 10]: TSTATE[0000004480001605] TPC[00000000006a54e0] 
TNPC[00000000006a54e4] TASK[swapper/10:0]
              TPC[atomic64_add+0x10/0x54] 
O7[__rb_reserve_next+0xc8/0x140] I7[rb_reserve_next_event+0xbc/0x220] 
RPC[ring_buffer_lock_reserve+0xd8/0x140]
   CPU[ 11]: TSTATE[0000000080001600] TPC[000000000042fcc0] 
TNPC[000000000042fcc4] TASK[swapper/11:0]
              TPC[stick_get_tick+0x0/0x20] O7[sched_clock+0x10/0x40] 
I7[trace_clock_local+0x4/0x60] RPC[rb_reserve_next_event+0x78/0x220]
   CPU[ 12]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/12:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 13]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/13:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 14]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/14:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 15]: TSTATE[0000001180001606] TPC[00000000004064ac] 
TNPC[00000000004064b0] TASK[abrt-action-sav:3155]
              TPC[sun4v_con_write+0xc/0x18] 
O7[sunhv_console_write_paged+0x130/0x1c0] 
I7[call_console_drivers.clone.1+0xe4/0x140] 
RPC[console_cont_flush.clone.0+0xc0/0x180]
   CPU[ 16]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/16:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 17]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/17:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 18]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/18:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 19]: TSTATE[0000004411001603] TPC[0000000000934fc8] 
TNPC[0000000000934fc0] TASK[login:3087]
              TPC[_raw_spin_trylock_bh+0xc8/0x100] 
O7[proc_clear_tty+0x34/0x60] I7[forget_original_parent+0x1c/0x1c0] 
RPC[exit_notify+0xc/0x160]
   CPU[ 20]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/20:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
* CPU[ 21]: TSTATE[0000000080001604] TPC[000000000042d5fc] 
TNPC[000000000042d600] TASK[swapper/21:0]
              TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 22]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/22:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 23]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/23:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 24]: TSTATE[0000004411001606] TPC[0000000000934f38] 
TNPC[0000000000934f3c] TASK[a.out:3145]
              TPC[_raw_spin_trylock_bh+0x38/0x100] 
O7[put_super+0x18/0x40] I7[grab_super_passive+0xc/0xa0] 
RPC[super_cache_count+0x10/0xc0]
   CPU[ 25]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/25:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 26]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/26:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 27]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/27:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 28]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/28:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 29]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/29:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 30]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/30:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 31]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/31:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 32]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/32:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 33]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/33:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 34]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/34:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 35]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/35:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 36]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/36:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 37]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/37:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 38]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/38:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 39]: TSTATE[0000004411001602] TPC[0000000000934f34] 
TNPC[0000000000934f38] TASK[auditd:2657]
              TPC[_raw_spin_trylock_bh+0x34/0x100] 
O7[put_super+0x18/0x40] I7[grab_super_passive+0xc/0xa0] 
RPC[super_cache_count+0x10/0xc0]
   CPU[ 40]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/40:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 41]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/41:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 42]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/42:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 43]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/43:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 44]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/44:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 45]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/45:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 46]: TSTATE[0000004480001604] TPC[000000000042d5a4] 
TNPC[000000000042d5a8] TASK[swapper/46:0]
              TPC[arch_cpu_idle+0x24/0xa0] O7[rcu_idle_enter+0x64/0xe0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 47]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/47:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 48]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/48:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 49]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/49:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 50]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/50:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 51]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/51:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 52]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/52:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 53]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/53:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 54]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/54:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 55]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/55:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 56]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/56:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 57]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/57:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 58]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/58:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 59]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/59:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 60]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/60:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 61]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/61:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 62]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/62:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]
   CPU[ 63]: TSTATE[0000000080001604] TPC[000000000042d5f8] 
TNPC[000000000042d5fc] TASK[swapper/63:0]
              TPC[arch_cpu_idle+0x78/0xa0] O7[arch_cpu_idle+0x64/0xa0] 
I7[cpu_idle_loop+0x13c/0x280] RPC[cpu_startup_entry+0x20/0x40]

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 10/03/2013 07:07 PM, Gurudas Pai wrote:
> On 09/30/2013 09:31 AM, David Miller wrote:
>> This is a respin of the original page_offset and 64-bit page
>> table changes with some minor changes and a bug fix:
>>
>> 1) Bob's ACKs for the first 6 patches added.
>>
>> 2) Don't use "UL" postfix for shift counts, there is code that
>>     wants those to be "int", particularly printfs in mm/mm_init.c
>>     that use the "%d" format specifier for SECTIONS_SHIFT.
>>
>> 3) Adjust VA_HOLE defines to match what the new 3-level page
>>     tables can actually provide.
>>
>> Signed-off-by: David S. Miller <davem@davemloft.net>
>> -- 
> On a t4-1 machine with 64gb RAM , run following testcase as
>
> ./a.out 10000
>
> And run echo 1000000 > /proc/sys/vm/nr_hugepages couple of times. 
> Machine becomes unusuable.
> This includes "fix generic hugetlb pte check return type." patch as 
> well.Oom killers and then lockups.
> Will try wothout patches very soon.
Same issue without 47-bit patches.
thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Bob Picco]

Hi,
David Miller wrote:	[Mon Sep 30 2013, 12:31:30PM EDT]
> 
> This is a respin of the original page_offset and 64-bit page
> table changes with some minor changes and a bug fix:
> 
> 1) Bob's ACKs for the first 6 patches added.
> 
> 2) Don't use "UL" postfix for shift counts, there is code that
>    wants those to be "int", particularly printfs in mm/mm_init.c
>    that use the "%d" format specifier for SECTIONS_SHIFT.
> 
> 3) Adjust VA_HOLE defines to match what the new 3-level page
>    tables can actually provide.
> 
> Signed-off-by: David S. Miller <davem@davemloft.net>
We seem to have our first issue with THP.

I first encountered it with "ld". A THP enabled value of "never" resolved the
issue.

Verified no issue on four level page table. Also no issue on x86_64 with
3.7.x and 3.11 with unmodified test code.

I've reviewed most of TT 0x31 paths. I commenced moving up in mm layer. 

Of course the debug code could have a bug :) I did examine some strace
output to check posix_memalign but not the glibc source.

It is Debian Wheezy and compiled with -m64. This on the T4-2 local to me.

The issue appears to be corruption.

thanx,

bob

<<CLIP HERE>>
#define	_GNU_SOURCE
#include <stdio.h>
#include <errno.h>
#include <stdlib.h>
#include <sys/mman.h>

#ifndef MADV_HUGEPAGE
#define MADV_HUGEPAGE	14
#endif

#define HPAGE_SHIFT	(22UL)
#define	HPAGE_SIZE	(1UL << HPAGE_SHIFT)
#define	PAGE_SHIFT	(13UL)
#define	NR_PAGES_HPAGE	(HPAGE_SIZE >> PAGE_SHIFT)
#define __round_mask(x, y) ((__typeof__(x))((y)-1))
#define round_up(x, y) ((((x)-1) | __round_mask(x, y))+1)

/* Note failure assumes an exit(1).
 */
void *allocate_thps(size_t sz)
{
	int rc;
	void *addr;
	size_t align = HPAGE_SIZE;

	rc = posix_memalign(&addr, align, sz);

	if (rc)
		perror("posix_memalign"), exit(1);

	rc = madvise(addr, sz, MADV_HUGEPAGE);

	if (rc)
		perror("madvise"), exit(1);

	return addr;
}

/* Verify the array.
 */
static void verify_thp(void *addr, void *end, int pagesize, int cnt)
{
	unsigned long pc;
	void *p;

	for (pc = 0, p = addr; p < end; p += pagesize, pc++) {
		if (*(unsigned long *) p != 0xbeefUL + (pc << 32UL)) {
			fprintf(stderr, "\n\tcnt = %d pc=0x%lx thp = 0x%lx "
				" addr=0x%lx *addr=0x%lx\n",
				cnt, pc, pc >> 9UL, p, *(unsigned long *) p);
			pc = (pc + NR_PAGES_HPAGE) & ~(NR_PAGES_HPAGE - 1);
			pc--;
			p = addr + (pc << PAGE_SHIFT);
		}
	}

}

int main(int argc, char **argv)
{
	void *addr, *end, *p;
	size_t sz;
	int pagesize = getpagesize();
	int loop, cnt, nthp, rc;
	int recycle = 1;
	unsigned long pc;
	int optind = 0;

	if (argc = 4 && argv[1][1] = 'a')
		optind = 1;
	else if (argc != 3)
		fprintf(stderr, "%s: -a iterations 4Mb-pages\n",
			argv[0]), exit(1);

	rc = sscanf(argv[optind + 1], "%d", &loop);

	if (rc != 1)
		fprintf(stderr, "%s: sscanf (%s) failed\n",
			argv[0], argv[optind + 1]), exit(1);

	rc = sscanf(argv[optind + 2], "%d", &nthp);

	if (rc != 1)
		fprintf(stderr, "%s: sscanf (%s) failed\n",
			argv[0], argv[optind + 2]), exit(1);

	if (optind)
		recycle = 0;

	sz = HPAGE_SIZE * nthp;

	for (cnt = 0; cnt < loop; cnt++) {
		addr = allocate_thps(sz);
		end = addr + sz;

		fprintf(stderr, "[0x%lx-0x%lx) ",
			(unsigned long) addr, (unsigned long) end);

		for (pc = 0, p = addr; p < end; p += pagesize, pc++)
			*(unsigned long *) p = 0xbeefUL + (pc << 32UL);

		verify_thp(addr, end, pagesize, cnt);

		sleep(cnt % 5);

		verify_thp(addr, end, pagesize, cnt);

		if (recycle)
			free(addr);

		fprintf(stdout, ".\n");
		fflush(stdout);
	}

	fprintf(stderr, "%s: Done! cnt = %d\n", argv[0], cnt);

	return 0;
}

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Bob Picco]

Hi,
David Miller wrote:	[Mon Sep 30 2013, 12:31:30PM EDT]
> 
> This is a respin of the original page_offset and 64-bit page
> table changes with some minor changes and a bug fix:
> 
> 1) Bob's ACKs for the first 6 patches added.
> 
> 2) Don't use "UL" postfix for shift counts, there is code that
>    wants those to be "int", particularly printfs in mm/mm_init.c
>    that use the "%d" format specifier for SECTIONS_SHIFT.
> 
> 3) Adjust VA_HOLE defines to match what the new 3-level page
>    tables can actually provide.
> 
> Signed-off-by: David S. Miller <davem@davemloft.net>

This issue I was deflected from by a wind direction change and never returned
to it. I believe you have been here with winfix_trampoline and collapsed
THP thrills. A large userspace stack will encounter a
[ 7603.877098] thp_stack[4583]: segfault at 100758 ip 0000000000100764 (rpc 0000000000100750) sp 000007fefdfff6d1 error 30001 in thp_stack[100000+2000]
.

For THP enabled set to "never", there is no issue. This could be a core
mm issue but not certain.

I'll attempt and find the time to return to this.

thanx,

bob

<<CLIP HERE>>
#define	_GNU_SOURCE
#include <stdio.h>
#include <errno.h>
#include <stdlib.h>

static unsigned long getfp()
{
	unsigned long fp;

	return  (unsigned long) &fp;
}

static void stack_grow(unsigned long start, unsigned long size, int bwait)
{
	unsigned long fp = getfp();
	unsigned long m = start - fp;
	unsigned long d;

	if (m > size) {
		fprintf(stderr, "at limit %lx\n", fp);
		if (bwait)
			for (d = 0; getfp() && d < (1UL <<30UL); d++);
		else
			sleep(5);
		return;
	} else 
		stack_grow(start, size, bwait);
}

void main(int argc, char **argv)
{
	unsigned long start = getfp();
	int loop, cnt, rc, bwait;
	unsigned long stk_sz;

	if (argc != 4)
		fprintf(stderr, "%s: stack-mb loop-count bwait\n", argv[0]),
			exit(1);

	rc = sscanf(argv[1], "%ld", &stk_sz);

	if (rc != 1)
		fprintf(stderr, "%s: stack-mb bad\n", argv[0]), exit(1);

	stk_sz <<= 20UL;

	rc = sscanf(argv[2], "%d", &loop);

	if (rc != 1)
		fprintf(stderr, "%s: loop count bad \n", argv[0]), exit(1);

	rc = sscanf(argv[3], "%d", &bwait);

	if (rc != 1)
		fprintf(stderr, "%s: bwait bad\n", argv[0]), exit(1);

	for (cnt = 0; cnt < loop; cnt++)
		stack_grow(start, stk_sz, bwait);
}

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Bob Picco <bpicco@meloft.net>
Date: Fri, 4 Oct 2013 11:45:55 -0400

> We seem to have our first issue with THP.
> 
> I first encountered it with "ld". A THP enabled value of "never" resolved the
> issue.
> 
> Verified no issue on four level page table. Also no issue on x86_64 with
> 3.7.x and 3.11 with unmodified test code.

You may want to see if reverting the last patch in my series makes the
problem go away, as a data point.

It's the trickiest change, as it changes the PMD layout completely.

If the problem persists, the next point to check is where the huge
page size gets adjusted to 8MB from 4MB.  Try reverted to just before
that change and test again.  That's the other major possible source of
errors.

Thanks.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Bob Picco]

Hi,
David Miller wrote:	[Fri Oct 04 2013, 02:00:59PM EDT]
> From: Bob Picco <bpicco@meloft.net>
> Date: Fri, 4 Oct 2013 11:45:55 -0400
> 
> > We seem to have our first issue with THP.
> > 
> > I first encountered it with "ld". A THP enabled value of "never" resolved the
> > issue.
> > 
> > Verified no issue on four level page table. Also no issue on x86_64 with
> > 3.7.x and 3.11 with unmodified test code.
> 
> You may want to see if reverting the last patch in my series makes the
> problem go away, as a data point.
I examined 9/9 very closely yesterday and under the tranquility of rain noise.
I didn't spot an issue. 
> 
> It's the trickiest change, as it changes the PMD layout completely.
> 
> If the problem persists, the next point to check is where the huge
> page size gets adjusted to 8MB from 4MB.  Try reverted to just before
> that change and test again.  That's the other major possible source of
> errors.
Yes, I suspected this and the reason for bottom up.

It seems to be here. With 9/9 and 8/9 reverted, the issue persists. Virgin
3.11 is solid. Of course there is always the possibility of mistake(s).

I've obviously missed an issue. Looking...

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Bob Picco <bpicco@meloft.net>
Date: Mon, 7 Oct 2013 12:07:45 -0400

> David Miller wrote:	[Fri Oct 04 2013, 02:00:59PM EDT]
>> It's the trickiest change, as it changes the PMD layout completely.
>> 
>> If the problem persists, the next point to check is where the huge
>> page size gets adjusted to 8MB from 4MB.  Try reverted to just before
>> that change and test again.  That's the other major possible source of
>> errors.
> Yes, I suspected this and the reason for bottom up.
> 
> It seems to be here. With 9/9 and 8/9 reverted, the issue persists. Virgin
> 3.11 is solid. Of course there is always the possibility of mistake(s).
> 
> I've obviously missed an issue. Looking...

Please, explicitly test both at the point where huge pages go from
4MB to 8MB and exactly right before that change.

Testing just a vanilla tree leaves too many variables in between.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Bob Picco]

Hi,
David Miller wrote:	[Mon Oct 07 2013, 12:08:50PM EDT]
> From: Bob Picco <bpicco@meloft.net>
> Date: Mon, 7 Oct 2013 12:07:45 -0400
> 
> > David Miller wrote:	[Fri Oct 04 2013, 02:00:59PM EDT]
> >> It's the trickiest change, as it changes the PMD layout completely.
> >> 
> >> If the problem persists, the next point to check is where the huge
> >> page size gets adjusted to 8MB from 4MB.  Try reverted to just before
> >> that change and test again.  That's the other major possible source of
> >> errors.
> > Yes, I suspected this and the reason for bottom up.
> > 
> > It seems to be here. With 9/9 and 8/9 reverted, the issue persists. Virgin
> > 3.11 is solid. Of course there is always the possibility of mistake(s).
> > 
> > I've obviously missed an issue. Looking...
> 
> Please, explicitly test both at the point where huge pages go from
> 4MB to 8MB and exactly right before that change.
I've done this on 3.12-rc4. The point (7/9) where we transition HPAGE_SHIFT
there are the issues already described with THP.
> 
> Testing just a vanilla tree leaves too many variables in between.
I agree.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Thu, 03 Oct 2013 19:07:11 -0700

> And run echo 1000000 > /proc/sys/vm/nr_hugepages couple of
> times.

You're running the machine out of memory and stressing the
setup path of the hugepage subsystem a lot.

This looks entirely different from the other crash/lockup,
in that there it looked like a cpu died with interrupts
disabled and thus wasn't responding even to cross calls.

This really isn't in a format I can look into, sorry.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Bob Picco <bpicco@meloft.net>
Date: Fri, 4 Oct 2013 11:45:55 -0400

> The issue appears to be corruption.

I think I know what the problem is.

In set_pmd_at() we need to flush two 4MB pages.

I'll play around with your test case and a potential fix, thanks.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Bob Picco]

David Miller wrote:	[Wed Oct 09 2013, 04:03:09PM EDT]
> From: Bob Picco <bpicco@meloft.net>
> Date: Fri, 4 Oct 2013 11:45:55 -0400
> 
> > The issue appears to be corruption.
> 
> I think I know what the problem is.
> 
> In set_pmd_at() we need to flush two 4MB pages.
> 
> I'll play around with your test case and a potential fix, thanks.
I believe you need this too. Otherwise the wrong tsb is computed.

 arch/sparc/mm/init_64.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c
index 4d0c99b..807e108 100644
--- a/arch/sparc/mm/init_64.c
+++ b/arch/sparc/mm/init_64.c
@@ -2722,7 +2722,7 @@ void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
 	spin_lock_irqsave(&mm->context.lock, flags);
 
 	if (mm->context.tsb_block[MM_TSB_HUGE].tsb != NULL)
-		__update_mmu_tsb_insert(mm, MM_TSB_HUGE, HPAGE_SHIFT,
+		__update_mmu_tsb_insert(mm, MM_TSB_HUGE, REAL_HPAGE_SHIFT,
 					addr, pte);
 
 	spin_unlock_irqrestore(&mm->context.lock, flags);

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 09/30/2013 09:31 AM, David Miller wrote:
> This is a respin of the original page_offset and 64-bit page
> table changes with some minor changes and a bug fix:
>
> 1) Bob's ACKs for the first 6 patches added.
>
> 2) Don't use "UL" postfix for shift counts, there is code that
>     wants those to be "int", particularly printfs in mm/mm_init.c
>     that use the "%d" format specifier for SECTIONS_SHIFT.
>
> 3) Adjust VA_HOLE defines to match what the new 3-level page
>     tables can actually provide.
>
> Signed-off-by: David S. Miller <davem@davemloft.net>
> --
> To unsubscribe from this list: send the line "unsubscribe sparclinux" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

This is on a  SPARC M10-1

PROMLIB: Sun IEEE Boot Prom 'OBP 4.34.0 2013/06/04 19:09'
PROMLIB: Root node compatible: sun4v
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 3.12.0-16mainline.sparc64 (root@xxx) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Oct 2 15:50:47 EDT 2013
bootconsole [earlyprom0] enabled
ARCH: SUN4V
Ethernet address: b0:99:28:9a:98:43
PAGE_OFFSET is 0xffff800000000000 (max_phys_bits = 47)
Kernel: Using 5 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /virtual-devices@100/console@1
PROM: Built device tree with 115806 bytes of memory.
MDESC: Size is 44112 bytes.
PLATFORM: banner-name [SPARC M10-1]
PLATFORM: name [ORCL,SPARC64-X]
PLATFORM: hostid [90060144]
PLATFORM: serial# [00000000]
PLATFORM: stick-frequency [05f5e100]
PLATFORM: mac-address [b099289a9843]
PLATFORM: watchdog-resolution [1000 ms]
PLATFORM: watchdog-max-timeout [31536000000 ms]
PLATFORM: max-cpus [32]
Top of RAM: 0x7e0ffffdc000, Total RAM: 0xf8f83e000
Memory hole size: 132122375MB
  [fc00800000-fc1fffffff] on node 0
Zone ranges:
   Normal   [mem 0x7e0070400000-0x7e0ffffdbfff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x7e0070400000-0x7e007fc9dfff]
   node   0: [mem 0x7e007fd40000-0x7e007fd5bfff]
   node   0: [mem 0x7e0080000000-0x7e0ffff61fff]
   node   0: [mem 0x7e0ffff72000-0x7e0ffff87fff]
   node   0: [mem 0x7e0ffffd0000-0x7e0ffffdbfff]
On node 0 totalpages: 8158239
   Normal zone: 63740 pages used for memmap
   Normal zone: 0 pages reserved
   Normal zone: 8158239 pages, LIFO batch:15
Booting Linux...
CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,ima]
PERCPU: Embedded 7 pages/cpu @fffffe0fe0000000 s23296 r8192 d25856 u131072
pcpu-alloc: s23296 r8192 d25856 u131072 alloc=1*4194304
pcpu-alloc: [0] 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
SUN4V: Mondo queue sizes [cpu(4096) dev(16384) r(8192) nr(256)]
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 8094499
Policy zone: Normal
Kernel command line: root=/dev/mapper/vg_m101370l0-lv_root ro rd_LVM_LV=vg_m101370l0/vol_swap rd_NO_LUKS rd_NO_MD KEYTABLE=us rd_LVM_LV=vg_m101370l0/lv_root LANG=ja_JP.UTF-8 rd_NO_DM
PID hash table entries: 4096 (order: 2, 32768 bytes)
Sorting __ex_table...
Memory: 64703096K/65265912K available (5333K kernel code, 708K rwdata, 2216K rodata, 488K init, 7896K bss, 562816K reserved)
Hierarchical RCU implementation.
	RCU restricting CPUs from NR_CPUSd to nr_cpu_ids2.
NR_IRQS:255
clocksource: mult[a000000] shift[24]
clockevent: mult[1999999a] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled, bootconsole disabled
allocated 132120576 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Calibrating delay using timer specific routine.. 200.06 BogoMIPS (lpj@0128)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Dentry cache hash table entries: 8388608 (order: 13, 67108864 bytes)
Inode-cache hash table entries: 4194304 (order: 12, 33554432 bytes)
Mount-cache hash table entries: 512
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
Initializing cgroup subsys hugetlb
ftrace: allocating 19188 entries in 38 pages
Performance events: No support for PMU type 'sparc64-x'
Brought up 32 CPUs
devtmpfs: initialized
ldc.c:v1.1 (July 22, 2008)
regulator-dummy: no parameters
NET: Registered protocol family 16
VIO: Adding device channel-devices
VIO: Adding device vldc-port-0-0
VIO: Adding device vldc-port-0-1
VIO: Adding device vldc-port-0-2
VIO: Adding device vldc-port-1-0
VIO: Adding device vldc-port-2-0
VIO: Adding device vldc-port-3-0
VIO: Adding device vlds-port-0-0
VIO: Adding device ds-0
VIO: Adding device ds-1
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
Missing cpus node, bailing out
pci_sun4v: Registered hvapi major[1] minor[0]
/pci@8000: SUN4V PCI Bus Module
/pci@8000: On NUMA node -1
/pci@8000: PCI IO[801100000000] MEM[800000000000]
/pci@8000: Unable to request IOMMU resource.
/pci@8000: Imported 2 TSB entries from OBP
/pci@8000: MSI Queue first[0] num[56] count[1024] devino[0x6]
/pci@8000: MSI first[0] num[1024] mask[0x3ff] width[32]
/pci@8000: MSI addr32[0x7f800000:0x800000] addr64[0xe00000003f800000:0x800000]
/pci@8000: MSI queues at RA [00007e0fd0800000]
PCI: Scanning PBM /pci@8000
pci_sun4v f0294a18: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x801100000000-0x80110fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007fffffff] (bus address [0x00000000-0x7fffffff])
pci_bus 0000:00: root bus resource [bus 00-08]
pci 0000:00:04.0: PME# supported from D0 D3hot D3cold
pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-08] (conflicts with (null) [bus 00-08])
pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
pci 0000:03:00.0: supports D1 D2
pci 0000:02:01.0: PME# supported from D0 D3hot D3cold
pci 0000:04:00.0: PME# supported from D0 D3hot
pci 0000:04:00.0: reg 0x184: [mem 0x110000000-0x110003fff 64bit pref]
pci 0000:04:00.0: reg 0x190: [mem 0x110020000-0x110023fff 64bit pref]
pci 0000:04:00.1: PME# supported from D0 D3hot
pci 0000:04:00.1: reg 0x184: [mem 0x110040000-0x110043fff 64bit pref]
pci 0000:04:00.1: reg 0x190: [mem 0x110060000-0x110063fff 64bit pref]
pci 0000:02:02.0: PME# supported from D0 D3hot D3cold
pci 0000:06:00.0: supports D1
pci 0000:06:00.0: PME# supported from D0 D1 D3hot
pci 0000:07:04.0: supports D1 D2
pci 0000:07:04.0: PME# supported from D0 D1 D2 D3hot
pci 0000:07:04.1: supports D1 D2
pci 0000:07:04.1: PME# supported from D0 D1 D2 D3hot
pci 0000:02:08.0: PME# supported from D0 D3hot D3cold
pci 0000:02:08.0: PCI bridge to [bus 08]
/pci@8100: SUN4V PCI Bus Module
/pci@8100: On NUMA node -1
/pci@8100: PCI IO[803100000000] MEM[802000000000]
/pci@8100: Unable to request IOMMU resource.
/pci@8100: MSI Queue first[0] num[56] count[1024] devino[0x6]
/pci@8100: MSI first[0] num[1024] mask[0x3ff] width[32]
/pci@8100: MSI addr32[0x7f800000:0x800000] addr64[0xe00000003f800000:0x800000]
/pci@8100: MSI queues at RA [00007e0fd0c00000]
PCI: Scanning PBM /pci@8100
pci_sun4v f02a0958: PCI host bridge to bus 0001:00
pci_bus 0001:00: root bus resource [io  0x803100000000-0x80310fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0001:00: root bus resource [mem 0x802000000000-0x80207fffffff] (bus address [0x00000000-0x7fffffff])
pci_bus 0001:00: root bus resource [bus 00-06]
pci 0001:00:04.0: PME# supported from D0 D3hot D3cold
pci_bus 0001:01: busn_res: can not insert [bus 01-ff] under [bus 00-06] (conflicts with (null) [bus 00-06])
pci 0001:01:00.0: PME# supported from D0 D3hot D3cold
pci 0001:02:00.0: PME# supported from D0 D3hot D3cold
pci 0001:03:00.0: PME# supported from D0 D3hot
pci 0001:03:00.0: reg 0x184: [mem 0x100000000-0x100003fff 64bit pref]
pci 0001:03:00.0: reg 0x190: [mem 0x100020000-0x100023fff 64bit pref]
pci 0001:03:00.1: PME# supported from D0 D3hot
pci 0001:03:00.1: reg 0x184: [mem 0x100040000-0x100043fff 64bit pref]
pci 0001:03:00.1: reg 0x190: [mem 0x100060000-0x100063fff 64bit pref]
pci 0001:02:01.0: PME# supported from D0 D3hot D3cold
pci 0001:02:09.0: PME# supported from D0 D3hot D3cold
bio: create slab <bio-0> at 0
vgaarb: loaded
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
SUN4V: Reboot data supported (maj=1,min=0).
ds.c:v1.0 (Jul 11, 2007)
Switched to clocksource stick
NET: Registered protocol family 2
TCP established hash table entries: 524288 (order: 10, 8388608 bytes)
TCP bind hash table entries: 65536 (order: 7, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP: reno registered
UDP hash table entries: 32768 (order: 7, 1048576 bytes)
UDP-Lite hash table entries: 32768 (order: 7, 1048576 bytes)
NET: Registered protocol family 1
PCI: CLS mismatch (64 != 512), using 64 bytes
PCI: Enabling device: (0000:00:04.0), cmd 407
PCI: Enabling device: (0000:07:04.1), cmd 2
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 26744K (fffffe0084000000 - fffffe0085a1e000)
Initialise module verification
audit: initializing netlink socket (disabled)
type 00 audit(1380862254.904:1): initialized
ds-1: Registered pri service.
HugeTLB registered 8 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 1024 (order 0, 8192 bytes)
msgmni has been set to 32768
SELinux:  Registering netfilter hooks
alg: No test for stdrng (krng)
Key type asymmetric registered
Asymmetric key parser 'x509' registered
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
io scheduler noop registered
io scheduler deadline registered (default)
io scheduler cfq registered
f0291a78: ttyS0 at I/O 0x0 (irq = 1, base_baud = 115200) is a SUN4V HCONS
ds-1: Registered var-config-backup service.
console [ttyHV0] enabled
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
brd: module loaded
loop: module loaded
libphy: Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-pci 0000:07:04.1: EHCI Host Controller
ehci-pci 0000:07:04.1: new USB bus registered, assigned bus number 1
ehci-pci 0000:07:04.1: irq 6, io mem 0x800000402000
ehci-pci 0000:07:04.1: USB 2.0 started, EHCI 1.00
usb usb1: New USB device found, idVendor\x1d6b, idProduct\002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 3.12.0-16mainline3.el6uek.sparc64 ehci_hcd
usb usb1: SerialNumber: 0000:07:04.1
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 3 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
ohci-pci 0000:07:04.0: OHCI PCI host controller
ohci-pci 0000:07:04.0: new USB bus registered, assigned bus number 2
ohci-pci 0000:07:04.0: irq 6, io mem 0x800000400000
usb usb2: New USB device found, idVendor\x1d6b, idProduct\001
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: OHCI PCI host controller
usb usb2: Manufacturer: Linux 3.12.0-16mainline3.el6uek.sparc64 ohci_hcd
usb usb2: SerialNumber: 0000:07:04.0
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 3 ports detected
uhci_hcd: USB Universal Host Controller Interface driver
mousedev: PS/2 mouse device common for all mice
rtc-sun4v rtc-sun4v: rtc core: registered sun4v as rtc0
hidraw: raw HID events driver (C) Jiri Kosina
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
zram: Created 1 device(s) ...
drop_monitor: Initializing network drop monitor service
TCP: cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Key type dns_resolver registered
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!
rtc-sun4v rtc-sun4v: setting system clock to 2013-10-04 04:50:57 UTC (1380862257)
dracut: dracut-004-303.0.9.el6
dracut: rd_NO_LUKS: removing cryptoluks activation
Loading iSCSI transport class v2.0-870.
iscsi: registered transport (qla4xxx)
QLogic iSCSI HBA Driver
libcxgbi:libcxgbi_init_module: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.
libcxgbi:ddp_setup_host_page_size: system PAGE 8192, ddp idx 1.
Chelsio T3 iSCSI Driver cxgb3i v2.0.0 (Jun. 2010)
iscsi: registered transport (cxgb3i)
Chelsio T4/T5 iSCSI Driver cxgb4i v0.9.4
iscsi: registered transport (cxgb4i)
NET: Registered protocol family 10
cnic: Broadcom NetXtreme II CNIC Driver cnic v2.5.18 (Sept 01, 2013)
Broadcom NetXtreme II iSCSI Driver bnx2i v2.7.6.2 (Jun 06, 2013)
iscsi: registered transport (bnx2i)
usb 1-2: new high-speed USB device number 2 using ehci-pci
iscsi: registered transport (be2iscsi)
In beiscsi_module_init, tt\0000000101fe960
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.26.0-ioctl (2013-08-15) initialised:dm-devel@redhat.com
udev: starting version 147
dracut: Starting plymouth daemon
usb 1-2: New USB device found, idVendor\x0411, idProduct\094
usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-2: Product: BUFFALO INC. USB2.0-IDE Bridge
usb 1-2: Manufacturer: BUFFALO INC.
usb 1-2: SerialNumber: 000000001C042125
usb-storage 1-2:1.0: USB Mass Storage device detected
scsi0 : usb-storage 1-2:1.0
usbcore: registered new interface driver usb-storage
dracut: rd_NO_DM: removing DM RAID activation
dracut: rd_NO_MD: removing MD RAID activation
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
scsi 0:0:0:0: CD-ROM            PIONEER  DVD-RW  DVR-112D 8.05 PQ: 0 ANSI: 0
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
camellia_sparc64: sparc64 camellia opcodes not available.
crc32c_sparc64: sparc64 crc32c opcode not available.
sha256_sparc64: sparc64 sha256 opcode not available.
aes_sparc64: sparc64 aes opcodes not available.
sha512_sparc64: sparc64 sha512 opcode not available.
des_sparc64: sparc64 des opcodes not available.
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti<giometti@linux.it>
PTP clock support registered
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.0.5-k
igb: Copyright (c) 2007-2013 Intel Corporation.
PCI: Enabling device: (0000:04:00.0), cmd 146
igb 0000:04:00.0: added PHC on eth0
igb 0000:04:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:04:00.0: eth0: (PCIe:2.5Gb/s:Width x2) b0:99:28:98:58:6c
igb 0000:04:00.0: eth0: PBA No: Unknown
igb 0000:04:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
PCI: Enabling device: (0000:04:00.1), cmd 146
igb 0000:04:00.1: added PHC on eth1
igb 0000:04:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:04:00.1: eth1: (PCIe:2.5Gb/s:Width x2) b0:99:28:98:58:6d
igb 0000:04:00.1: eth1: PBA No: Unknown
igb 0000:04:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
PCI: Enabling device: (0001:00:04.0), cmd 407
PCI: Enabling device: (0001:03:00.0), cmd 146
igb 0001:03:00.0: added PHC on eth2
igb 0001:03:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0001:03:00.0: eth2: (PCIe:2.5Gb/s:Width x2) b0:99:28:98:58:6e
igb 0001:03:00.0: eth2: PBA No: Unknown
igb 0001:03:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
PCI: Enabling device: (0001:03:00.1), cmd 146
igb 0001:03:00.1: added PHC on eth3
igb 0001:03:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0001:03:00.1: eth3: (PCIe:2.5Gb/s:Width x2) b0:99:28:98:58:6f
igb 0001:03:00.1: eth3: PBA No: Unknown
igb 0001:03:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
mpt2sas version 16.100.00.00 loaded
scsi1 : Fusion MPT SAS Host
mpt2sas0: 32 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (64730328 kB)
mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 179
mpt2sas0-msix1: PCI-MSI-X enabled: IRQ 180
mpt2sas0-msix2: PCI-MSI-X enabled: IRQ 181
mpt2sas0-msix3: PCI-MSI-X enabled: IRQ 182
mpt2sas0-msix4: PCI-MSI-X enabled: IRQ 183
mpt2sas0-msix5: PCI-MSI-X enabled: IRQ 184
mpt2sas0-msix6: PCI-MSI-X enabled: IRQ 185
mpt2sas0-msix7: PCI-MSI-X enabled: IRQ 186
mpt2sas0-msix8: PCI-MSI-X enabled: IRQ 187
mpt2sas0-msix9: PCI-MSI-X enabled: IRQ 188
mpt2sas0-msix10: PCI-MSI-X enabled: IRQ 189
mpt2sas0-msix11: PCI-MSI-X enabled: IRQ 190
mpt2sas0-msix12: PCI-MSI-X enabled: IRQ 191
mpt2sas0-msix13: PCI-MSI-X enabled: IRQ 192
mpt2sas0-msix14: PCI-MSI-X enabled: IRQ 193
mpt2sas0-msix15: PCI-MSI-X enabled: IRQ 194
mpt2sas0: iomem(0x0000800100000000), mapped(0x0000800100000000), size(65536)
mpt2sas0: ioport(0x0000801100000000), size(256)
mpt2sas0: Allocated physical memory: size(4992 kB)
mpt2sas0: Current Controller Queue Depth(2811), Max Controller Queue Depth(3072)
mpt2sas0: Scatter Gather Elements per IO(128)
mpt2sas0: LSISAS2308: FWVersion(13.00.66.00), ChipRevision(0x05), BiosVersion(00.00.00.00)
mpt2sas0: Protocol=(Initiator), Capabilities=(Raid,TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
mpt2sas0: sending port enable !!
mpt2sas0: host_add: handle(0x0001), sas_addr(0x500000e0e0450000), phys(8)
mpt2sas0: expander_add: handle(0x0009), parent(0x0001), sas_addr(0x500000e0e049113f), phys(22)
mpt2sas0: port enable: SUCCESS
scsi 1:0:0:0: Direct-Access     TOSHIBA  MBF2600RC        3706 PQ: 0 ANSI: 5
scsi 1:0:0:0: SSP: handle(0x000a), sas_addr(0x50000394281b5b92), phy(4), device_name(0x50000394281b5b91)
scsi 1:0:0:0: SSP: enclosure_logical_id(0x500000e0e049113f), slot(4)
scsi 1:0:0:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(6), cmd_que(1)
scsi 1:0:1:0: Enclosure         FUJITSU  NBBEXP           0d32 PQ: 0 ANSI: 5
scsi 1:0:1:0: SSP: handle(0x000b), sas_addr(0x500000e0e049113d), phy(20), device_name(0x0000000000000000)
scsi 1:0:1:0: SSP: enclosure_logical_id(0x500000e0e049113f), slot(0)
scsi 1:0:1:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(6), cmd_que(1)
niu: niu.c:v1.1 (Apr 22, 2010)
niu: niu0: Found PHY 002060b1 type MII at phy_port 10
niu: niu0: Found PHY 002060b1 type MII at phy_port 11
niu: niu0: Found PHY 002060b1 type MII at phy_port 12
niu: niu0: Found PHY 002060b1 type MII at phy_port 13
niu: niu0: Port 0 [4 RX chans] [6 TX chans]
niu: niu0: Port 1 [4 RX chans] [6 TX chans]
niu: niu0: Port 2 [4 RX chans] [6 TX chans]
niu: niu0: Port 3 [4 RX chans] [6 TX chans]
niu: niu0: Port 0 RDC tbl(0) [ 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 ]
niu: niu0: Port 0 RDC tbl(1) [ 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 ]
niu: niu0: Port 1 RDC tbl(2) [ 4 5 6 7 4 5 6 7 4 5 6 7 4 5 6 7 ]
niu: niu0: Port 1 RDC tbl(3) [ 4 5 6 7 4 5 6 7 4 5 6 7 4 5 6 7 ]
niu: niu0: Port 2 RDC tbl(4) [ 8 9 10 11 8 9 10 11 8 9 10 11 8 9 10 11 ]
niu: niu0: Port 2 RDC tbl(5) [ 8 9 10 11 8 9 10 11 8 9 10 11 8 9 10 11 ]
niu: niu0: Port 3 RDC tbl(6) [ 12 13 14 15 12 13 14 15 12 13 14 15 12 13 14 15 ]
niu: niu0: Port 3 RDC tbl(7) [ 12 13 14 15 12 13 14 15 12 13 14 15 12 13 14 15 ]
niu: eth4: NIU Ethernet 00:10:e0:10:8b:04
niu: eth4: Port type[XMAC] mode[1G:COPPER] XCVR[MII] phy[mif]
niu: eth5: NIU Ethernet 00:10:e0:10:8b:05
niu: eth5: Port type[XMAC] mode[1G:COPPER] XCVR[MII] phy[mif]
niu: eth6: NIU Ethernet 00:10:e0:10:8b:06
niu: eth6: Port type[BMAC] mode[1G:COPPER] XCVR[MII] phy[mif]
niu: eth7: NIU Ethernet 00:10:e0:10:8b:07
niu: eth7: Port type[BMAC] mode[1G:COPPER] XCVR[MII] phy[mif]
qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.06.00.08-k.
qla2xxx [0001:06:00.0]-001d: : Found an ISP2532 irq 10 iobase 0x0000802120000000.
scsi2 : qla2xxx
qla2xxx [0001:06:00.0]-00fb:2: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA.
qla2xxx [0001:06:00.0]-00fc:2: ISP2532: PCIe (5.0GT/s x8) @ 0001:06:00.0 hdma- host#=2 fw=7.01.00 (90d5).
qla2xxx [0001:06:00.1]-001d: : Found an ISP2532 irq 11 iobase 0x0000802120004000.
qla2xxx [0001:06:00.0]-505f:2: Link is operational (4 Gbps).
qla2xxx [0001:06:00.0]-1020:2: **** Failed mbx[0]@06, mb[1]\x7fe, mb[2]=1, mb[3]=3, cmdj ****.
scsi 2:0:0:0: Direct-Access     FUJITSU  ETERNUS_DXL      0000 PQ: 0 ANSI: 5
scsi 2:0:0:1: Direct-Access     FUJITSU  ETERNUS_DXL      0000 PQ: 0 ANSI: 5
scsi3 : qla2xxx
qla2xxx [0001:06:00.1]-00fb:3: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA.
qla2xxx [0001:06:00.1]-00fc:3: ISP2532: PCIe (5.0GT/s x8) @ 0001:06:00.1 hdma- host#=3 fw=7.01.00 (90d5).
sr0: scsi3-mmc drive: 62x/62x writer cd/rw xa/form2 cdda tray
cdrom: Uniform CD-ROM driver Revision: 3.20
sr 0:0:0:0: Attached scsi CD-ROM sr0
sd 1:0:0:0: [sda] 1172123568 512-byte logical blocks: (600 GB/558 GiB)
sd 1:0:0:0: [sda] Write Protect is off
sd 1:0:0:0: [sda] Mode Sense: d3 00 00 08
sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  sda: sda1 sda2 sda3
sd 1:0:0:0: [sda] Attached SCSI disk
sd 2:0:0:0: [sdb] 2048000 512-byte logical blocks: (1.04 GB/1000 MiB)
sd 2:0:0:1: [sdc] 2048000 512-byte logical blocks: (1.04 GB/1000 MiB)
sd 2:0:0:1: [sdc] Write Protect is off
sd 2:0:0:1: [sdc] Mode Sense: 8f 00 00 08
sd 2:0:0:1: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
  sdc: sdc1 sdc2 sdc3 sdc7
sd 2:0:0:1: [sdc] Attached SCSI disk
sd 2:0:0:0: [sdb] Write Protect is off
sd 2:0:0:0: [sdb] Mode Sense: 8f 00 00 08
sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
  sdb: sdb1 sdb2 sdb3 sdb7
sd 2:0:0:0: [sdb] Attached SCSI disk
dracut: Scanning devices sda2  for LVM logical volumes vg_m101370l0/vol_swap vg_m101370l0/lv_root
dracut: inactive '/dev/vg_m101370l0/lv_root' [362.59 GiB] inherit
dracut: inactive '/dev/vg_m101370l0/vol_swap' [195.31 GiB] inherit
bio: create slab <bio-1> at 1
EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
dracut: Mounted root filesystem /dev/mapper/vg_m101370l0-lv_root
dracut: Loading SELinux policy
type\x1404 audit(1380862286.356:2): enforcing=1 old_enforcing=0 auidB94967295 sesB94967295
SELinux: 2048 avtab hash slots, 276422 rules.
SELinux: 2048 avtab hash slots, 276422 rules.
SELinux:  9 users, 12 roles, 3890 types, 201 bools, 1 sens, 1024 cats
SELinux:  81 classes, 276422 rules
SELinux:  Permission wake_alarm in class capability2 not defined in policy.
SELinux:  Permission block_suspend in class capability2 not defined in policy.
SELinux:  Permission attach_queue in class tun_socket not defined in policy.
SELinux: the above unknown classes and permissions will be allowed
SELinux:  Completing initialization.
SELinux:  Setting up existing superblocks.
SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
SELinux: initialized (dev proc, type proc), uses genfs_contexts
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev devtmpfs, type devtmpfs), uses transition SIDs
SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts
SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
SELinux: initialized (dev anon_inodefs, type anon_inodefs), uses genfs_contexts
SELinux: initialized (dev devpts, type devpts), uses transition SIDs
SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses transition SIDs
SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
SELinux: initialized (dev securityfs, type securityfs), uses genfs_contexts
SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev dm-0, type ext4), uses xattr
type\x1403 audit(1380862287.036:3): policy loaded auidB94967295 sesB94967295
dracut:
dracut: Switching root
readahead-collector: starting
readahead-collector: sorting
readahead-collector: finished
udev: starting version 147
EXT4-fs (dm-0): re-mounted. Opts: (null)
Adding 204799992k swap on /dev/mapper/vg_m101370l0-vol_swap.  Priority:-1 extents:1 across:204799992k FS
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses genfs_contexts
IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
qla2xxx [0001:06:00.1]-8038:3: Cable is unplugged...
rsyslogd (2041): Attempt to access syslog with CAP_SYS_ADMIN but no CAP_SYSLOG (deprecated).
IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready
IPv6: ADDRCONF(NETDEV_UP): eth4: link is not ready
IPv6: ADDRCONF(NETDEV_UP): eth5: link is not ready
IPv6: ADDRCONF(NETDEV_UP): eth6: link is not ready
IPv6: ADDRCONF(NETDEV_UP): eth7: link is not ready
igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Fri, 11 Oct 2013 14:26:52 -0700

> CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
> CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,ima]

Hmmm, lots of capabilities missing I thought would be there.
For example, all the crypto opcodes.

Are they really not there in the T5 chips?  Indeed they don't
show up in the hwcap-list of the cpu MDESC node :-/

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 10/11/2013 02:54 PM, David Miller wrote:
> From: Gurudas Pai <gurudas.pai@oracle.com>
> Date: Fri, 11 Oct 2013 14:26:52 -0700
>
>> CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
>> CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,ima]
> Hmmm, lots of capabilities missing I thought would be there.
> For example, all the crypto opcodes.
>
> Are they really not there in the T5 chips?  Indeed they don't
> show up in the hwcap-list of the cpu MDESC node :-/
This from sparc64-x, not t5.
Thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Fri, 11 Oct 2013 19:05:40 -0700

> On 10/11/2013 02:54 PM, David Miller wrote:
>> From: Gurudas Pai <gurudas.pai@oracle.com>
>> Date: Fri, 11 Oct 2013 14:26:52 -0700
>>
>>> CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
>>> CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,ima]
>> Hmmm, lots of capabilities missing I thought would be there.
>> For example, all the crypto opcodes.
>>
>> Are they really not there in the T5 chips?  Indeed they don't
>> show up in the hwcap-list of the cpu MDESC node :-/
> This from sparc64-x, not t5.

Oh ok, that makes more sense, thanks.

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[Gurudas Pai]

On 10/09/2013 12:18 PM, David Miller wrote:
> From: Gurudas Pai <gurudas.pai@oracle.com>
> Date: Thu, 03 Oct 2013 19:07:11 -0700
>
>> And run echo 1000000 > /proc/sys/vm/nr_hugepages couple of
>> times.
> You're running the machine out of memory and stressing the
> setup path of the hugepage subsystem a lot.
>
> This looks entirely different from the other crash/lockup,
> in that there it looked like a cpu died with interrupts
> disabled and thus wasn't responding even to cross calls.
>
> This really isn't in a format I can look into, sorry.
Following change fixes the issue for me, (this was removed by
759496ba6407c6994d6a5ce3a5e74937d7816208), not very sure how.

diff -Nrup arch/sparc/mm/fault_64.c /tmp/fault_64.c
--- arch/sparc/mm/fault_64.c    2013-10-22 20:10:04.302411223 -0400
+++ /tmp/fault_64.c    2013-10-22 20:10:19.593410899 -0400
@@ -427,6 +427,7 @@ good_area:
              goto bad_area;
      }

+    flags |= ((fault_code & FAULT_CODE_WRITE) ? FAULT_FLAG_WRITE : 0);
      fault = handle_mm_fault(mm, vma, address, flags);

      if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current))


Thanks,
-Guru

Re: [PATCH v2 0/9] sparc64: Support 47-bit physical addresses.

[David Miller]

From: Gurudas Pai <gurudas.pai@oracle.com>
Date: Tue, 22 Oct 2013 18:18:18 -0700

> On 10/09/2013 12:18 PM, David Miller wrote:
>> From: Gurudas Pai <gurudas.pai@oracle.com>
>> Date: Thu, 03 Oct 2013 19:07:11 -0700
>>
>>> And run echo 1000000 > /proc/sys/vm/nr_hugepages couple of
>>> times.
>> You're running the machine out of memory and stressing the
>> setup path of the hugepage subsystem a lot.
>>
>> This looks entirely different from the other crash/lockup,
>> in that there it looked like a cpu died with interrupts
>> disabled and thus wasn't responding even to cross calls.
>>
>> This really isn't in a format I can look into, sorry.
> Following change fixes the issue for me, (this was removed by
> 759496ba6407c6994d6a5ce3a5e74937d7816208), not very sure how.
> 
> diff -Nrup arch/sparc/mm/fault_64.c /tmp/fault_64.c
> --- arch/sparc/mm/fault_64.c    2013-10-22 20:10:04.302411223 -0400
> +++ /tmp/fault_64.c    2013-10-22 20:10:19.593410899 -0400
> @@ -427,6 +427,7 @@ good_area:
>              goto bad_area;
>      }
> 
> + flags |= ((fault_code & FAULT_CODE_WRITE) ? FAULT_FLAG_WRITE : 0);
>      fault = handle_mm_fault(mm, vma, address, flags);
> 
>      if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current))

The code block right above that sets the write flag when necessary.

I think you're just changing the timing and avoiding a race of some
sort, it's the only thing that makes any sense.