* arcmsr & areca-1660 - strange behaviour under heavy load
@ 2008-02-23 11:20 Nikola Ciprich
2008-02-25 0:10 ` Andrew Morton
0 siblings, 1 reply; 8+ messages in thread
From: Nikola Ciprich @ 2008-02-23 11:20 UTC (permalink / raw)
To: linux-kernel
Hi,
I've found strange problem either in arcmsr driver, or maybe in
areca-1660 card...
When system on SAS discs RAID connected to areca-1660 card
gets under heavy I/O load, it gets unusable after some time. I can 100% reproduce
this, although it needs quite speciffic conditions:
It can be reproduced on 2x quad core machine, RAM has to be limited to
~192MB to cause heavy paging.
Only thing needed to cause the problem is to start loop doing kernel
compilation using make -j 8 - this loads the system heavily, because of
lack of memory. After few correct compile runs the system gets into
state when all programs including the basic ones (ls, cp, ..) start
crashing... dmesg (when it works) doesn't say anything strange...
After reboot, the system is OK again.
I have tested it on different motherboards, with different CPUs, RAMs(all
were properly tested with memtest), with two different areca cards and
different drives. I can't reproduce the problem on same hardware when
using different RAID card (ie adaptec). All testing systems were properly
cooled..
I have tried all available areca firmwares, two different distributions
(oracle linux, and centos), and kernels ranging from distribution ones, to last GIT snapshot.
Could somebody please give me some hints on how to hunt this problem?
Areca support doesn't seem to be very interested in the problem :-(
Thanks a lot in advance
BR
nik
-------------------------------------
Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax: +420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: servis@linuxbox.cz
-------------------------------------
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: arcmsr & areca-1660 - strange behaviour under heavy load
2008-02-23 11:20 arcmsr & areca-1660 - strange behaviour under heavy load Nikola Ciprich
@ 2008-02-25 0:10 ` Andrew Morton
2008-02-26 9:35 ` Nikola Ciprich
0 siblings, 1 reply; 8+ messages in thread
From: Andrew Morton @ 2008-02-25 0:10 UTC (permalink / raw)
To: Nikola Ciprich; +Cc: linux-kernel, linux-scsi, Nick Cheng, Erich Chen
On Sat, 23 Feb 2008 12:20:12 +0100 (CET) Nikola Ciprich <extmaillist@linuxbox.cz> wrote:
> Hi,
>
> I've found strange problem either in arcmsr driver, or maybe in
> areca-1660 card...
> When system on SAS discs RAID connected to areca-1660 card
> gets under heavy I/O load, it gets unusable after some time. I can 100% reproduce
> this, although it needs quite speciffic conditions:
> It can be reproduced on 2x quad core machine, RAM has to be limited to
> ~192MB to cause heavy paging.
> Only thing needed to cause the problem is to start loop doing kernel
> compilation using make -j 8 - this loads the system heavily, because of
> lack of memory. After few correct compile runs the system gets into
> state when all programs including the basic ones (ls, cp, ..) start
> crashing... dmesg (when it works) doesn't say anything strange...
> After reboot, the system is OK again.
> I have tested it on different motherboards, with different CPUs, RAMs(all
> were properly tested with memtest), with two different areca cards and
> different drives. I can't reproduce the problem on same hardware when
> using different RAID card (ie adaptec). All testing systems were properly
> cooled..
> I have tried all available areca firmwares, two different distributions
> (oracle linux, and centos), and kernels ranging from distribution ones, to last GIT snapshot.
> Could somebody please give me some hints on how to hunt this problem?
> Areca support doesn't seem to be very interested in the problem :-(
(cc's added)
Please get the machine into this state of memory exhaustion then take
copies of the output of the following, and send them via reply-to-all to
this email:
- cat /proc/meminfo
- cat /proc/slabinfo
- dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
Thanks.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: arcmsr & areca-1660 - strange behaviour under heavy load
2008-02-25 0:10 ` Andrew Morton
@ 2008-02-26 9:35 ` Nikola Ciprich
2008-02-26 10:30 ` nickcheng
2008-02-26 17:43 ` Andrew Morton
0 siblings, 2 replies; 8+ messages in thread
From: Nikola Ciprich @ 2008-02-26 9:35 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, linux-scsi, Nick Cheng, Erich Chen, kopi
[-- Attachment #1: Type: TEXT/PLAIN, Size: 861 bytes --]
Hi
On Sun, 24 Feb 2008, Andrew Morton wrote:
Hi Andrew,
thanks a lot for reply, I'm attaching requested information.
please let me know if You need more information/testing, whatever.
I'll be glad to help.
BR
nik
>> Areca support doesn't seem to be very interested in the problem :-(
>
> (cc's added)
>
> Please get the machine into this state of memory exhaustion then take
> copies of the output of the following, and send them via reply-to-all to
> this email:
>
> - cat /proc/meminfo
>
> - cat /proc/slabinfo
>
> - dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
>
> Thanks.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
--
[-- Attachment #2: Type: TEXT/plain, Size: 777 bytes --]
MemTotal: 188464 kB
MemFree: 82268 kB
Buffers: 20572 kB
Cached: 40312 kB
SwapCached: 1508 kB
Active: 35220 kB
Inactive: 29436 kB
SwapTotal: 3145712 kB
SwapFree: 3142156 kB
Dirty: 340 kB
Writeback: 0 kB
AnonPages: 2836 kB
Mapped: 4532 kB
Slab: 29824 kB
SReclaimable: 16728 kB
SUnreclaim: 13096 kB
PageTables: 1024 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 3239944 kB
Committed_AS: 13732 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 10644 kB
VmallocChunk: 34359727343 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
[-- Attachment #3: Type: TEXT/plain, Size: 16223 bytes --]
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
fib6_nodes 5 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
ip6_dst_cache 4 12 320 12 1 : tunables 54 27 8 : slabdata 1 1 0
ndisc_cache 1 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
RAWv6 11 12 896 4 1 : tunables 54 27 8 : slabdata 3 3 0
UDPLITEv6 0 0 896 4 1 : tunables 54 27 8 : slabdata 0 0 0
UDPv6 0 0 896 4 1 : tunables 54 27 8 : slabdata 0 0 0
tw_sock_TCPv6 0 0 192 20 1 : tunables 120 60 8 : slabdata 0 0 0
request_sock_TCPv6 0 0 192 20 1 : tunables 120 60 8 : slabdata 0 0 0
TCPv6 2 4 1664 4 2 : tunables 24 12 8 : slabdata 1 1 0
ip_fib_alias 10 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
ip_fib_hash 10 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
reiser_inode_cache 13 15 776 5 1 : tunables 54 27 8 : slabdata 3 3 0
dm_mpath_io 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0
dm_snap_pending_exception 128 136 112 34 1 : tunables 120 60 8 : slabdata 4 4 0
dm_snap_exception 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0
dm_uevent 0 0 2608 3 2 : tunables 24 12 8 : slabdata 0 0 0
dm_target_io 1320 1440 24 144 1 : tunables 120 60 8 : slabdata 10 10 0
dm_io 1320 1472 40 92 1 : tunables 120 60 8 : slabdata 16 16 0
scsi_cmd_cache 38 40 384 10 1 : tunables 54 27 8 : slabdata 4 4 0
sgpool-128 2 2 4096 1 1 : tunables 24 12 8 : slabdata 2 2 0
sgpool-64 2 2 2048 2 1 : tunables 24 12 8 : slabdata 1 1 0
sgpool-32 2 4 1024 4 1 : tunables 54 27 8 : slabdata 1 1 0
sgpool-16 3 8 512 8 1 : tunables 54 27 8 : slabdata 1 1 0
sgpool-8 13 45 256 15 1 : tunables 120 60 8 : slabdata 3 3 0
scsi_io_context 0 0 112 34 1 : tunables 120 60 8 : slabdata 0 0 0
ext3_inode_cache 3946 4028 832 4 1 : tunables 54 27 8 : slabdata 1007 1007 0
ext3_xattr 0 0 88 44 1 : tunables 120 60 8 : slabdata 0 0 0
journal_handle 32 144 24 144 1 : tunables 120 60 8 : slabdata 1 1 0
journal_head 105 280 96 40 1 : tunables 120 60 8 : slabdata 7 7 0
revoke_table 6 202 16 202 1 : tunables 120 60 8 : slabdata 1 1 0
revoke_record 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0
uhci_urb_priv 0 0 56 67 1 : tunables 120 60 8 : slabdata 0 0 0
UNIX 8 33 704 11 2 : tunables 54 27 8 : slabdata 3 3 0
flow_cache 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
cfq_io_context 20 175 152 25 1 : tunables 120 60 8 : slabdata 7 7 0
cfq_queue 15 112 136 28 1 : tunables 120 60 8 : slabdata 4 4 0
mqueue_inode_cache 1 4 896 4 1 : tunables 54 27 8 : slabdata 1 1 0
hugetlbfs_inode_cache 1 6 608 6 1 : tunables 54 27 8 : slabdata 1 1 0
ext2_inode_cache 1 4 824 4 1 : tunables 54 27 8 : slabdata 1 1 0
ext2_xattr 0 0 88 44 1 : tunables 120 60 8 : slabdata 0 0 0
dnotify_cache 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0
dquot 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0
inotify_event_cache 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0
inotify_watch_cache 1 53 72 53 1 : tunables 120 60 8 : slabdata 1 1 0
kioctx 0 0 320 12 1 : tunables 54 27 8 : slabdata 0 0 0
kiocb 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0
fasync_cache 0 0 24 144 1 : tunables 120 60 8 : slabdata 0 0 0
shmem_inode_cache 823 905 816 5 1 : tunables 54 27 8 : slabdata 181 181 0
nsproxy 0 0 56 67 1 : tunables 120 60 8 : slabdata 0 0 0
posix_timers_cache 0 0 160 24 1 : tunables 120 60 8 : slabdata 0 0 0
uid_cache 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0
ip_mrt_cache 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
UDP-Lite 0 0 768 5 1 : tunables 54 27 8 : slabdata 0 0 0
tcp_bind_bucket 1 112 32 112 1 : tunables 120 60 8 : slabdata 1 1 0
inet_peer_cache 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0
secpath_cache 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0
xfrm_dst_cache 0 0 384 10 1 : tunables 54 27 8 : slabdata 0 0 0
ip_dst_cache 22 40 384 10 1 : tunables 54 27 8 : slabdata 4 4 0
arp_cache 3 30 256 15 1 : tunables 120 60 8 : slabdata 2 2 0
RAW 9 11 704 11 2 : tunables 54 27 8 : slabdata 1 1 0
UDP 2 10 768 5 1 : tunables 54 27 8 : slabdata 2 2 0
tw_sock_TCP 0 0 192 20 1 : tunables 120 60 8 : slabdata 0 0 0
request_sock_TCP 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
TCP 0 0 1536 5 2 : tunables 24 12 8 : slabdata 0 0 0
eventpoll_pwq 0 0 72 53 1 : tunables 120 60 8 : slabdata 0 0 0
eventpoll_epi 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
blkdev_ioc 20 177 64 59 1 : tunables 120 60 8 : slabdata 3 3 0
blkdev_queue 25 36 1656 4 2 : tunables 24 12 8 : slabdata 9 9 0
blkdev_requests 22 52 288 13 1 : tunables 54 27 8 : slabdata 2 4 0
biovec-256 82 82 4096 1 1 : tunables 24 12 8 : slabdata 82 82 0
biovec-128 82 82 2048 2 1 : tunables 24 12 8 : slabdata 41 41 0
biovec-64 82 84 1024 4 1 : tunables 54 27 8 : slabdata 21 21 0
biovec-16 82 135 256 15 1 : tunables 120 60 8 : slabdata 9 9 0
biovec-4 82 354 64 59 1 : tunables 120 60 8 : slabdata 6 6 0
biovec-1 121 404 16 202 1 : tunables 120 60 8 : slabdata 2 2 0
bio 118 240 128 30 1 : tunables 120 60 8 : slabdata 7 8 0
sock_inode_cache 44 70 704 5 1 : tunables 54 27 8 : slabdata 14 14 0
skbuff_fclone_cache 28 28 512 7 1 : tunables 54 27 8 : slabdata 4 4 0
skbuff_head_cache 333 390 256 15 1 : tunables 120 60 8 : slabdata 26 26 0
file_lock_cache 2 44 176 22 1 : tunables 120 60 8 : slabdata 2 2 0
Acpi-Operand 1060 1475 64 59 1 : tunables 120 60 8 : slabdata 25 25 0
Acpi-ParseExt 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-Parse 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-State 0 0 80 48 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-Namespace 744 784 32 112 1 : tunables 120 60 8 : slabdata 7 7 0
task_delay_info 141 708 64 59 1 : tunables 120 60 8 : slabdata 12 12 0
taskstats 0 0 312 12 1 : tunables 54 27 8 : slabdata 0 0 0
proc_inode_cache 648 672 640 6 1 : tunables 54 27 8 : slabdata 112 112 0
sigqueue 0 24 160 24 1 : tunables 120 60 8 : slabdata 0 1 0
radix_tree_node 2429 2947 552 7 1 : tunables 54 27 8 : slabdata 421 421 0
bdev_cache 26 40 832 4 1 : tunables 54 27 8 : slabdata 10 10 0
sysfs_dir_cache 9212 9312 80 48 1 : tunables 120 60 8 : slabdata 194 194 0
mnt_cache 28 90 256 15 1 : tunables 120 60 8 : slabdata 6 6 0
inode_cache 125 150 608 6 1 : tunables 54 27 8 : slabdata 25 25 0
dentry 39043 39159 200 19 1 : tunables 120 60 8 : slabdata 2061 2061 0
filp 271 1340 192 20 1 : tunables 120 60 8 : slabdata 67 67 0
names_cache 2 2 4096 1 1 : tunables 24 12 8 : slabdata 2 2 0
avc_node 31 106 72 53 1 : tunables 120 60 8 : slabdata 2 2 0
selinux_inode_security 5612 8200 96 40 1 : tunables 120 60 8 : slabdata 205 205 0
idr_layer_cache 120 140 528 7 1 : tunables 54 27 8 : slabdata 20 20 0
buffer_head 33945 34669 104 37 1 : tunables 120 60 8 : slabdata 937 937 0
mm_struct 52 108 832 9 2 : tunables 54 27 8 : slabdata 12 12 0
vm_area_struct 801 2553 168 23 1 : tunables 120 60 8 : slabdata 111 111 0
fs_cache 42 354 64 59 1 : tunables 120 60 8 : slabdata 6 6 0
files_cache 43 70 768 5 1 : tunables 54 27 8 : slabdata 14 14 0
signal_cache 140 252 832 9 2 : tunables 54 27 8 : slabdata 28 28 0
sighand_cache 136 174 2112 3 2 : tunables 24 12 8 : slabdata 58 58 0
task_struct 136 162 1984 2 1 : tunables 24 12 8 : slabdata 81 81 0
anon_vma 309 1008 24 144 1 : tunables 120 60 8 : slabdata 7 7 0
pid_namespace 0 0 2104 3 2 : tunables 24 12 8 : slabdata 0 0 0
pid_1 135 600 128 30 1 : tunables 120 60 8 : slabdata 20 20 0
size-4194304(DMA) 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0
size-4194304 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0
size-2097152(DMA) 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0
size-2097152 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0
size-1048576(DMA) 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0
size-1048576 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0
size-524288(DMA) 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0
size-524288 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0
size-262144(DMA) 0 0 262144 1 64 : tunables 1 1 0 : slabdata 0 0 0
size-262144 0 0 262144 1 64 : tunables 1 1 0 : slabdata 0 0 0
size-131072(DMA) 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-131072 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-65536(DMA) 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-65536 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-32768(DMA) 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-32768 4 4 32768 1 8 : tunables 8 4 0 : slabdata 4 4 0
size-16384(DMA) 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0
size-16384 15 15 16384 1 4 : tunables 8 4 0 : slabdata 15 15 0
size-8192(DMA) 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0
size-8192 13 13 8192 1 2 : tunables 8 4 0 : slabdata 13 13 0
size-4096(DMA) 0 0 4096 1 1 : tunables 24 12 8 : slabdata 0 0 0
size-4096 221 221 4096 1 1 : tunables 24 12 8 : slabdata 221 221 0
size-2048(DMA) 0 0 2048 2 1 : tunables 24 12 8 : slabdata 0 0 0
size-2048 512 530 2048 2 1 : tunables 24 12 8 : slabdata 265 265 0
size-1024(DMA) 0 0 1024 4 1 : tunables 54 27 8 : slabdata 0 0 0
size-1024 1458 1472 1024 4 1 : tunables 54 27 8 : slabdata 367 368 0
size-512(DMA) 0 0 512 8 1 : tunables 54 27 8 : slabdata 0 0 0
size-512 542 592 512 8 1 : tunables 54 27 8 : slabdata 74 74 0
size-256(DMA) 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0
size-256 888 1005 256 15 1 : tunables 120 60 8 : slabdata 67 67 0
size-128(DMA) 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
size-64(DMA) 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0
size-64 2470 5133 64 59 1 : tunables 120 60 8 : slabdata 87 87 0
size-32(DMA) 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0
size-128 1385 1560 128 30 1 : tunables 120 60 8 : slabdata 52 52 0
size-32 7730 8064 32 112 1 : tunables 120 60 8 : slabdata 72 72 0
kmem_cache 147 160 384 10 1 : tunables 54 27 8 : slabdata 16 16 0
[-- Attachment #4: Type: TEXT/plain, Size: 2824 bytes --]
[ 4863.335825] SysRq : Show Memory
[ 4863.335949] Mem-info:
[ 4863.335952] DMA per-cpu:
[ 4863.335954] CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 4863.336008] CPU 1: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 4863.336010] CPU 2: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 4863.336012] CPU 3: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 4863.336014] CPU 4: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 4863.336016] CPU 5: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 4863.336018] CPU 6: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 4863.336019] CPU 7: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 4863.336021] DMA32 per-cpu:
[ 4863.336022] CPU 0: Hot: hi: 42, btch: 7 usd: 44 Cold: hi: 14, btch: 3 usd: 4
[ 4863.336024] CPU 1: Hot: hi: 42, btch: 7 usd: 17 Cold: hi: 14, btch: 3 usd: 2
[ 4863.336026] CPU 2: Hot: hi: 42, btch: 7 usd: 10 Cold: hi: 14, btch: 3 usd: 1
[ 4863.336028] CPU 3: Hot: hi: 42, btch: 7 usd: 27 Cold: hi: 14, btch: 3 usd: 3
[ 4863.336030] CPU 4: Hot: hi: 42, btch: 7 usd: 35 Cold: hi: 14, btch: 3 usd: 2
[ 4863.336032] CPU 5: Hot: hi: 42, btch: 7 usd: 20 Cold: hi: 14, btch: 3 usd: 1
[ 4863.336034] CPU 6: Hot: hi: 42, btch: 7 usd: 36 Cold: hi: 14, btch: 3 usd: 2
[ 4863.336035] CPU 7: Hot: hi: 42, btch: 7 usd: 36 Cold: hi: 14, btch: 3 usd: 3
[ 4863.336038] Active:8805 inactive:7364 dirty:6 writeback:0 unstable:0
[ 4863.336039] free:20589 slab:7401 mapped:1133 pagetables:247 bounce:0
[ 4863.336041] DMA free:10764kB min:100kB low:124kB high:148kB active:344kB inactive:184kB present:11292kB pages_scanned:0 all_unreclaimable? no
[ 4863.336043] lowmem_reserve[]: 0 173 173 173
[ 4863.336046] DMA32 free:71592kB min:1632kB low:2040kB high:2448kB active:34876kB inactive:29272kB present:177760kB pages_scanned:0 all_unreclaimable? no
[ 4863.336048] lowmem_reserve[]: 0 0 0 0
[ 4863.336050] DMA: 211*4kB 194*8kB 147*16kB 88*32kB 34*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 10764kB
[ 4863.336056] DMA32: 486*4kB 488*8kB 1373*16kB 842*32kB 235*64kB 10*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 71592kB
[ 4863.336391] Swap cache: add 20537385, delete 20539682, find 2655799/6042712, race 157+192
[ 4863.336393] Free swap = 3142156kB
[ 4863.336395] Total swap = 3145712kB
[ 4863.336396] Free swap: 3142156kB
[ 4863.337203] 49152 pages of RAM
[ 4863.337206] 2036 reserved pages
[ 4863.337209] 11197 pages shared
[ 4863.337210] 377 pages swap cached
^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: arcmsr & areca-1660 - strange behaviour under heavy load
2008-02-26 9:35 ` Nikola Ciprich
@ 2008-02-26 10:30 ` nickcheng
2008-02-26 17:43 ` Andrew Morton
1 sibling, 0 replies; 8+ messages in thread
From: nickcheng @ 2008-02-26 10:30 UTC (permalink / raw)
To: 'Nikola Ciprich', 'Andrew Morton'
Cc: linux-kernel, linux-scsi, 'Erich Chen', kopi, kevin34,
billion.wu
Hi Nikola,
As I said, we will test on our site.
Our support team will help you to settle the issue.
Sorry for your inconvenience,
-----Original Message-----
From: Nikola Ciprich [mailto:extmaillist@linuxbox.cz]
Sent: Tuesday, February 26, 2008 5:36 PM
To: Andrew Morton
Cc: linux-kernel@vger.kernel.org; linux-scsi@vger.kernel.org; Nick Cheng;
Erich Chen; kopi@linuxbox.cz
Subject: Re: arcmsr & areca-1660 - strange behaviour under heavy load
Hi
On Sun, 24 Feb 2008, Andrew Morton wrote:
Hi Andrew,
thanks a lot for reply, I'm attaching requested information.
please let me know if You need more information/testing, whatever.
I'll be glad to help.
BR
nik
>> Areca support doesn't seem to be very interested in the problem :-(
>
> (cc's added)
>
> Please get the machine into this state of memory exhaustion then take
> copies of the output of the following, and send them via reply-to-all to
> this email:
>
> - cat /proc/meminfo
>
> - cat /proc/slabinfo
>
> - dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
>
> Thanks.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
--
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: arcmsr & areca-1660 - strange behaviour under heavy load
2008-02-26 9:35 ` Nikola Ciprich
2008-02-26 10:30 ` nickcheng
@ 2008-02-26 17:43 ` Andrew Morton
2008-02-26 19:29 ` Nikola Ciprich
1 sibling, 1 reply; 8+ messages in thread
From: Andrew Morton @ 2008-02-26 17:43 UTC (permalink / raw)
To: Nikola Ciprich; +Cc: linux-kernel, linux-scsi, Nick Cheng, Erich Chen, kopi
On Tue, 26 Feb 2008 10:35:31 +0100 (CET) Nikola Ciprich <extmaillist@linuxbox.cz> wrote:
> Hi
>
> On Sun, 24 Feb 2008, Andrew Morton wrote:
>
> Hi Andrew,
> thanks a lot for reply, I'm attaching requested information.
> please let me know if You need more information/testing, whatever.
> I'll be glad to help.
> BR
> nik
>
> >> Areca support doesn't seem to be very interested in the problem :-(
> >
> > (cc's added)
> >
> > Please get the machine into this state of memory exhaustion then take
> > copies of the output of the following, and send them via reply-to-all to
> > this email:
> >
> > - cat /proc/meminfo
> >
> > - cat /proc/slabinfo
> >
> > - dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
> >
> > Thanks.
Alas, that all looks OK to me.
You never get any out-of-memory messages, and no oom-killing messages?
Possibly what is happening here is that in this low-memory condition, some
of the driver's internal memory-allocation attempts are failing, and the
driver isn't correctly handling this. This is a rare situation which may
well not have been hit in anyone else's testing.
I expect that the Areca engineers will be able to reproduce this with a
suitably small "mem=" kernel boot option. If not, they could perhaps
investigate the kernel's fault-injection framework, which permits
simulation of page allocation failures.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: arcmsr & areca-1660 - strange behaviour under heavy load
2008-02-26 17:43 ` Andrew Morton
@ 2008-02-26 19:29 ` Nikola Ciprich
2008-02-26 21:04 ` Zan Lynx
0 siblings, 1 reply; 8+ messages in thread
From: Nikola Ciprich @ 2008-02-26 19:29 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, linux-scsi, Nick Cheng, Erich Chen, kopi
Hi Andrew,
no, right now I have the machine in the weird state, swap is empty (3GB),
and so is bigger part of RAM (~100MB free), and the gcc crashes even when
trying to compile c program with empty main function. so it doesn't seem
to be problem with memory exhaustion.
Hopefully the areca guys will be able to find out what is going on. But
anyways, if You'll have any other idea what should I check/try, please let
me know, as I have to admit that I'd really like to hunt it down myself
(and yes, there is some vanity on my side here :))
thanks a lot once more
cheers
nik
On Tue, 26 Feb 2008,
Andrew Morton wrote:
> On Tue, 26 Feb 2008 10:35:31 +0100 (CET) Nikola Ciprich <extmaillist@linuxbox.cz> wrote:
>
>> Hi
>>
>> On Sun, 24 Feb 2008, Andrew Morton wrote:
>>
>> Hi Andrew,
>> thanks a lot for reply, I'm attaching requested information.
>> please let me know if You need more information/testing, whatever.
>> I'll be glad to help.
>> BR
>> nik
>>
>>>> Areca support doesn't seem to be very interested in the problem :-(
>>>
>>> (cc's added)
>>>
>>> Please get the machine into this state of memory exhaustion then take
>>> copies of the output of the following, and send them via reply-to-all to
>>> this email:
>>>
>>> - cat /proc/meminfo
>>>
>>> - cat /proc/slabinfo
>>>
>>> - dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
>>>
>>> Thanks.
>
> Alas, that all looks OK to me.
>
> You never get any out-of-memory messages, and no oom-killing messages?
>
> Possibly what is happening here is that in this low-memory condition, some
> of the driver's internal memory-allocation attempts are failing, and the
> driver isn't correctly handling this. This is a rare situation which may
> well not have been hit in anyone else's testing.
>
> I expect that the Areca engineers will be able to reproduce this with a
> suitably small "mem=" kernel boot option. If not, they could perhaps
> investigate the kernel's fault-injection framework, which permits
> simulation of page allocation failures.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
--
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: arcmsr & areca-1660 - strange behaviour under heavy load
2008-02-26 19:29 ` Nikola Ciprich
@ 2008-02-26 21:04 ` Zan Lynx
2008-02-27 1:53 ` nickcheng
0 siblings, 1 reply; 8+ messages in thread
From: Zan Lynx @ 2008-02-26 21:04 UTC (permalink / raw)
To: Nikola Ciprich
Cc: Andrew Morton, linux-kernel, linux-scsi, Nick Cheng, Erich Chen,
kopi
[-- Attachment #1: Type: text/plain, Size: 833 bytes --]
On Tue, 2008-02-26 at 20:29 +0100, Nikola Ciprich wrote:
> Hi Andrew,
> no, right now I have the machine in the weird state, swap is empty (3GB),
> and so is bigger part of RAM (~100MB free), and the gcc crashes even when
> trying to compile c program with empty main function. so it doesn't seem
> to be problem with memory exhaustion.
Maybe memory fragmentation? Perhaps the driver tries to allocate a
large block of memory and cannot find a continuous block of the right
size.
Maybe the driver developers used different kernel .config options than
you are using.
Try increasing the value in /proc/sys/vm/min_free_kbytes.
Try switching some things like SLAB or SLUB, try booting with
kernelcore=512M to enable the Movable memory zone, or try 64-bit vs
32-bit kernels.
--
Zan Lynx <zlynx@acm.org>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: arcmsr & areca-1660 - strange behaviour under heavy load
2008-02-26 21:04 ` Zan Lynx
@ 2008-02-27 1:53 ` nickcheng
0 siblings, 0 replies; 8+ messages in thread
From: nickcheng @ 2008-02-27 1:53 UTC (permalink / raw)
To: 'Nikola Ciprich'
Cc: 'Andrew Morton', linux-kernel, linux-scsi,
'Erich Chen', kopi, support, 'Zan Lynx'
Hi Nikola,
Please put support@areca.com.tw in the loop.
I am sure Areca support, Kevin, has taken over your case.
If you like, please let him know your configuration and operations to
synchronize both sides.
Thank you for your patience and sorry for your inconvenience,
-----Original Message-----
From: Zan Lynx [mailto:zlynx@acm.org]
Sent: Wednesday, February 27, 2008 5:04 AM
To: Nikola Ciprich
Cc: Andrew Morton; linux-kernel@vger.kernel.org; linux-scsi@vger.kernel.org;
Nick Cheng; Erich Chen; kopi@linuxbox.cz
Subject: Re: arcmsr & areca-1660 - strange behaviour under heavy load
On Tue, 2008-02-26 at 20:29 +0100, Nikola Ciprich wrote:
> Hi Andrew,
> no, right now I have the machine in the weird state, swap is empty (3GB),
> and so is bigger part of RAM (~100MB free), and the gcc crashes even when
> trying to compile c program with empty main function. so it doesn't seem
> to be problem with memory exhaustion.
Maybe memory fragmentation? Perhaps the driver tries to allocate a
large block of memory and cannot find a continuous block of the right
size.
Maybe the driver developers used different kernel .config options than
you are using.
Try increasing the value in /proc/sys/vm/min_free_kbytes.
Try switching some things like SLAB or SLUB, try booting with
kernelcore=512M to enable the Movable memory zone, or try 64-bit vs
32-bit kernels.
--
Zan Lynx <zlynx@acm.org>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2008-02-27 1:53 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-02-23 11:20 arcmsr & areca-1660 - strange behaviour under heavy load Nikola Ciprich
2008-02-25 0:10 ` Andrew Morton
2008-02-26 9:35 ` Nikola Ciprich
2008-02-26 10:30 ` nickcheng
2008-02-26 17:43 ` Andrew Morton
2008-02-26 19:29 ` Nikola Ciprich
2008-02-26 21:04 ` Zan Lynx
2008-02-27 1:53 ` nickcheng
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox