arcmsr & areca-1660 - strange behaviour under heavy load

public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed

* arcmsr & areca-1660 - strange behaviour under heavy load
@ 2008-02-23 11:20 Nikola Ciprich
  2008-02-25  0:10 ` Andrew Morton
  0 siblings, 1 reply; 8+ messages in thread
From: Nikola Ciprich @ 2008-02-23 11:20 UTC (permalink / raw)
  To: linux-kernel

Hi,

I've found strange problem either in arcmsr driver, or maybe in 
areca-1660 card...
When system on SAS discs RAID connected to areca-1660 card 
gets under heavy I/O load, it gets unusable after some time. I can 100% reproduce 
this, although it needs quite speciffic conditions:
It can be reproduced on 2x quad core machine, RAM has to be limited to 
~192MB to cause heavy paging.
Only thing needed to cause the problem is to start loop doing kernel 
compilation using make -j 8 - this loads the system heavily, because of 
lack of memory. After few correct compile runs the system gets into 
state when all programs including the basic ones (ls, cp, ..) start 
crashing... dmesg (when it works) doesn't say anything strange...
After reboot, the system is OK again.
I have tested it on different motherboards, with different CPUs, RAMs(all 
were properly tested with memtest), with two different areca cards and 
different drives. I can't reproduce the problem on same hardware when 
using different RAID card (ie adaptec). All testing systems were properly 
cooled..
I have tried all available areca firmwares, two different distributions 
(oracle linux, and centos), and kernels ranging from distribution ones, to last GIT snapshot.
Could somebody please give me some hints on how to hunt this problem?
Areca support doesn't seem to be very interested in the problem :-(
Thanks a lot in advance
BR
nik

-------------------------------------
Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava

tel.:   +420 596 603 142
fax:    +420 596 621 273
mobil:  +420 777 093 799

www.linuxbox.cz

mobil servis: +420 737 238 656
email servis: servis@linuxbox.cz
-------------------------------------

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: arcmsr & areca-1660 - strange behaviour under heavy load
  2008-02-23 11:20 arcmsr & areca-1660 - strange behaviour under heavy load Nikola Ciprich
@ 2008-02-25  0:10 ` Andrew Morton
  2008-02-26  9:35   ` Nikola Ciprich
  0 siblings, 1 reply; 8+ messages in thread
From: Andrew Morton @ 2008-02-25  0:10 UTC (permalink / raw)
  To: Nikola Ciprich; +Cc: linux-kernel, linux-scsi, Nick Cheng, Erich Chen

On Sat, 23 Feb 2008 12:20:12 +0100 (CET) Nikola Ciprich <extmaillist@linuxbox.cz> wrote:

> Hi,
> 
> I've found strange problem either in arcmsr driver, or maybe in 
> areca-1660 card...
> When system on SAS discs RAID connected to areca-1660 card 
> gets under heavy I/O load, it gets unusable after some time. I can 100% reproduce 
> this, although it needs quite speciffic conditions:
> It can be reproduced on 2x quad core machine, RAM has to be limited to 
> ~192MB to cause heavy paging.
> Only thing needed to cause the problem is to start loop doing kernel 
> compilation using make -j 8 - this loads the system heavily, because of 
> lack of memory. After few correct compile runs the system gets into 
> state when all programs including the basic ones (ls, cp, ..) start 
> crashing... dmesg (when it works) doesn't say anything strange...
> After reboot, the system is OK again.
> I have tested it on different motherboards, with different CPUs, RAMs(all 
> were properly tested with memtest), with two different areca cards and 
> different drives. I can't reproduce the problem on same hardware when 
> using different RAID card (ie adaptec). All testing systems were properly 
> cooled..
> I have tried all available areca firmwares, two different distributions 
> (oracle linux, and centos), and kernels ranging from distribution ones, to last GIT snapshot.
> Could somebody please give me some hints on how to hunt this problem?
> Areca support doesn't seem to be very interested in the problem :-(

(cc's added)

Please get the machine into this state of memory exhaustion then take
copies of the output of the following, and send them via reply-to-all to
this email:

- cat /proc/meminfo

- cat /proc/slabinfo

- dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c

Thanks.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: arcmsr & areca-1660 - strange behaviour under heavy load
  2008-02-25  0:10 ` Andrew Morton
@ 2008-02-26  9:35   ` Nikola Ciprich
  2008-02-26 10:30     ` nickcheng
  2008-02-26 17:43     ` Andrew Morton
  0 siblings, 2 replies; 8+ messages in thread
From: Nikola Ciprich @ 2008-02-26  9:35 UTC (permalink / raw)
  To: Andrew Morton; +Cc: linux-kernel, linux-scsi, Nick Cheng, Erich Chen, kopi

[-- Attachment #1: Type: TEXT/PLAIN, Size: 861 bytes --]

Hi

On Sun, 24 Feb 2008, Andrew Morton wrote:

Hi Andrew,
thanks a lot for reply, I'm attaching requested information.
please let me know if You need more information/testing, whatever.
I'll be glad to help.
BR
nik

>> Areca support doesn't seem to be very interested in the problem :-(
>
> (cc's added)
>
> Please get the machine into this state of memory exhaustion then take
> copies of the output of the following, and send them via reply-to-all to
> this email:
>
> - cat /proc/meminfo
>
> - cat /proc/slabinfo
>
> - dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
>
> Thanks.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>

-- 

[-- Attachment #2: Type: TEXT/plain, Size: 777 bytes --]

MemTotal:       188464 kB
MemFree:         82268 kB
Buffers:         20572 kB
Cached:          40312 kB
SwapCached:       1508 kB
Active:          35220 kB
Inactive:        29436 kB
SwapTotal:     3145712 kB
SwapFree:      3142156 kB
Dirty:             340 kB
Writeback:           0 kB
AnonPages:        2836 kB
Mapped:           4532 kB
Slab:            29824 kB
SReclaimable:    16728 kB
SUnreclaim:      13096 kB
PageTables:       1024 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3239944 kB
Committed_AS:    13732 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     10644 kB
VmallocChunk: 34359727343 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
HugePages_Surp:      0
Hugepagesize:     2048 kB

[-- Attachment #3: Type: TEXT/plain, Size: 16223 bytes --]

slabinfo - version: 2.1
# name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
fib6_nodes             5     59     64   59    1 : tunables  120   60    8 : slabdata      1      1      0
ip6_dst_cache          4     12    320   12    1 : tunables   54   27    8 : slabdata      1      1      0
ndisc_cache            1     15    256   15    1 : tunables  120   60    8 : slabdata      1      1      0
RAWv6                 11     12    896    4    1 : tunables   54   27    8 : slabdata      3      3      0
UDPLITEv6              0      0    896    4    1 : tunables   54   27    8 : slabdata      0      0      0
UDPv6                  0      0    896    4    1 : tunables   54   27    8 : slabdata      0      0      0
tw_sock_TCPv6          0      0    192   20    1 : tunables  120   60    8 : slabdata      0      0      0
request_sock_TCPv6      0      0    192   20    1 : tunables  120   60    8 : slabdata      0      0      0
TCPv6                  2      4   1664    4    2 : tunables   24   12    8 : slabdata      1      1      0
ip_fib_alias          10     59     64   59    1 : tunables  120   60    8 : slabdata      1      1      0
ip_fib_hash           10     59     64   59    1 : tunables  120   60    8 : slabdata      1      1      0
reiser_inode_cache     13     15    776    5    1 : tunables   54   27    8 : slabdata      3      3      0
dm_mpath_io            0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
dm_snap_pending_exception    128    136    112   34    1 : tunables  120   60    8 : slabdata      4      4      0
dm_snap_exception      0      0     32  112    1 : tunables  120   60    8 : slabdata      0      0      0
dm_uevent              0      0   2608    3    2 : tunables   24   12    8 : slabdata      0      0      0
dm_target_io        1320   1440     24  144    1 : tunables  120   60    8 : slabdata     10     10      0
dm_io               1320   1472     40   92    1 : tunables  120   60    8 : slabdata     16     16      0
scsi_cmd_cache        38     40    384   10    1 : tunables   54   27    8 : slabdata      4      4      0
sgpool-128             2      2   4096    1    1 : tunables   24   12    8 : slabdata      2      2      0
sgpool-64              2      2   2048    2    1 : tunables   24   12    8 : slabdata      1      1      0
sgpool-32              2      4   1024    4    1 : tunables   54   27    8 : slabdata      1      1      0
sgpool-16              3      8    512    8    1 : tunables   54   27    8 : slabdata      1      1      0
sgpool-8              13     45    256   15    1 : tunables  120   60    8 : slabdata      3      3      0
scsi_io_context        0      0    112   34    1 : tunables  120   60    8 : slabdata      0      0      0
ext3_inode_cache    3946   4028    832    4    1 : tunables   54   27    8 : slabdata   1007   1007      0
ext3_xattr             0      0     88   44    1 : tunables  120   60    8 : slabdata      0      0      0
journal_handle        32    144     24  144    1 : tunables  120   60    8 : slabdata      1      1      0
journal_head         105    280     96   40    1 : tunables  120   60    8 : slabdata      7      7      0
revoke_table           6    202     16  202    1 : tunables  120   60    8 : slabdata      1      1      0
revoke_record          0      0     32  112    1 : tunables  120   60    8 : slabdata      0      0      0
uhci_urb_priv          0      0     56   67    1 : tunables  120   60    8 : slabdata      0      0      0
UNIX                   8     33    704   11    2 : tunables   54   27    8 : slabdata      3      3      0
flow_cache             0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
cfq_io_context        20    175    152   25    1 : tunables  120   60    8 : slabdata      7      7      0
cfq_queue             15    112    136   28    1 : tunables  120   60    8 : slabdata      4      4      0
mqueue_inode_cache      1      4    896    4    1 : tunables   54   27    8 : slabdata      1      1      0
hugetlbfs_inode_cache      1      6    608    6    1 : tunables   54   27    8 : slabdata      1      1      0
ext2_inode_cache       1      4    824    4    1 : tunables   54   27    8 : slabdata      1      1      0
ext2_xattr             0      0     88   44    1 : tunables  120   60    8 : slabdata      0      0      0
dnotify_cache          0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
dquot                  0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
inotify_event_cache      0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
inotify_watch_cache      1     53     72   53    1 : tunables  120   60    8 : slabdata      1      1      0
kioctx                 0      0    320   12    1 : tunables   54   27    8 : slabdata      0      0      0
kiocb                  0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
fasync_cache           0      0     24  144    1 : tunables  120   60    8 : slabdata      0      0      0
shmem_inode_cache    823    905    816    5    1 : tunables   54   27    8 : slabdata    181    181      0
nsproxy                0      0     56   67    1 : tunables  120   60    8 : slabdata      0      0      0
posix_timers_cache      0      0    160   24    1 : tunables  120   60    8 : slabdata      0      0      0
uid_cache              0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
ip_mrt_cache           0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
UDP-Lite               0      0    768    5    1 : tunables   54   27    8 : slabdata      0      0      0
tcp_bind_bucket        1    112     32  112    1 : tunables  120   60    8 : slabdata      1      1      0
inet_peer_cache        0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
secpath_cache          0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
xfrm_dst_cache         0      0    384   10    1 : tunables   54   27    8 : slabdata      0      0      0
ip_dst_cache          22     40    384   10    1 : tunables   54   27    8 : slabdata      4      4      0
arp_cache              3     30    256   15    1 : tunables  120   60    8 : slabdata      2      2      0
RAW                    9     11    704   11    2 : tunables   54   27    8 : slabdata      1      1      0
UDP                    2     10    768    5    1 : tunables   54   27    8 : slabdata      2      2      0
tw_sock_TCP            0      0    192   20    1 : tunables  120   60    8 : slabdata      0      0      0
request_sock_TCP       0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
TCP                    0      0   1536    5    2 : tunables   24   12    8 : slabdata      0      0      0
eventpoll_pwq          0      0     72   53    1 : tunables  120   60    8 : slabdata      0      0      0
eventpoll_epi          0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
blkdev_ioc            20    177     64   59    1 : tunables  120   60    8 : slabdata      3      3      0
blkdev_queue          25     36   1656    4    2 : tunables   24   12    8 : slabdata      9      9      0
blkdev_requests       22     52    288   13    1 : tunables   54   27    8 : slabdata      2      4      0
biovec-256            82     82   4096    1    1 : tunables   24   12    8 : slabdata     82     82      0
biovec-128            82     82   2048    2    1 : tunables   24   12    8 : slabdata     41     41      0
biovec-64             82     84   1024    4    1 : tunables   54   27    8 : slabdata     21     21      0
biovec-16             82    135    256   15    1 : tunables  120   60    8 : slabdata      9      9      0
biovec-4              82    354     64   59    1 : tunables  120   60    8 : slabdata      6      6      0
biovec-1             121    404     16  202    1 : tunables  120   60    8 : slabdata      2      2      0
bio                  118    240    128   30    1 : tunables  120   60    8 : slabdata      7      8      0
sock_inode_cache      44     70    704    5    1 : tunables   54   27    8 : slabdata     14     14      0
skbuff_fclone_cache     28     28    512    7    1 : tunables   54   27    8 : slabdata      4      4      0
skbuff_head_cache    333    390    256   15    1 : tunables  120   60    8 : slabdata     26     26      0
file_lock_cache        2     44    176   22    1 : tunables  120   60    8 : slabdata      2      2      0
Acpi-Operand        1060   1475     64   59    1 : tunables  120   60    8 : slabdata     25     25      0
Acpi-ParseExt          0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
Acpi-Parse             0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
Acpi-State             0      0     80   48    1 : tunables  120   60    8 : slabdata      0      0      0
Acpi-Namespace       744    784     32  112    1 : tunables  120   60    8 : slabdata      7      7      0
task_delay_info      141    708     64   59    1 : tunables  120   60    8 : slabdata     12     12      0
taskstats              0      0    312   12    1 : tunables   54   27    8 : slabdata      0      0      0
proc_inode_cache     648    672    640    6    1 : tunables   54   27    8 : slabdata    112    112      0
sigqueue               0     24    160   24    1 : tunables  120   60    8 : slabdata      0      1      0
radix_tree_node     2429   2947    552    7    1 : tunables   54   27    8 : slabdata    421    421      0
bdev_cache            26     40    832    4    1 : tunables   54   27    8 : slabdata     10     10      0
sysfs_dir_cache     9212   9312     80   48    1 : tunables  120   60    8 : slabdata    194    194      0
mnt_cache             28     90    256   15    1 : tunables  120   60    8 : slabdata      6      6      0
inode_cache          125    150    608    6    1 : tunables   54   27    8 : slabdata     25     25      0
dentry             39043  39159    200   19    1 : tunables  120   60    8 : slabdata   2061   2061      0
filp                 271   1340    192   20    1 : tunables  120   60    8 : slabdata     67     67      0
names_cache            2      2   4096    1    1 : tunables   24   12    8 : slabdata      2      2      0
avc_node              31    106     72   53    1 : tunables  120   60    8 : slabdata      2      2      0
selinux_inode_security   5612   8200     96   40    1 : tunables  120   60    8 : slabdata    205    205      0
idr_layer_cache      120    140    528    7    1 : tunables   54   27    8 : slabdata     20     20      0
buffer_head        33945  34669    104   37    1 : tunables  120   60    8 : slabdata    937    937      0
mm_struct             52    108    832    9    2 : tunables   54   27    8 : slabdata     12     12      0
vm_area_struct       801   2553    168   23    1 : tunables  120   60    8 : slabdata    111    111      0
fs_cache              42    354     64   59    1 : tunables  120   60    8 : slabdata      6      6      0
files_cache           43     70    768    5    1 : tunables   54   27    8 : slabdata     14     14      0
signal_cache         140    252    832    9    2 : tunables   54   27    8 : slabdata     28     28      0
sighand_cache        136    174   2112    3    2 : tunables   24   12    8 : slabdata     58     58      0
task_struct          136    162   1984    2    1 : tunables   24   12    8 : slabdata     81     81      0
anon_vma             309   1008     24  144    1 : tunables  120   60    8 : slabdata      7      7      0
pid_namespace          0      0   2104    3    2 : tunables   24   12    8 : slabdata      0      0      0
pid_1                135    600    128   30    1 : tunables  120   60    8 : slabdata     20     20      0
size-4194304(DMA)      0      0 4194304    1 1024 : tunables    1    1    0 : slabdata      0      0      0
size-4194304           0      0 4194304    1 1024 : tunables    1    1    0 : slabdata      0      0      0
size-2097152(DMA)      0      0 2097152    1  512 : tunables    1    1    0 : slabdata      0      0      0
size-2097152           0      0 2097152    1  512 : tunables    1    1    0 : slabdata      0      0      0
size-1048576(DMA)      0      0 1048576    1  256 : tunables    1    1    0 : slabdata      0      0      0
size-1048576           0      0 1048576    1  256 : tunables    1    1    0 : slabdata      0      0      0
size-524288(DMA)       0      0 524288    1  128 : tunables    1    1    0 : slabdata      0      0      0
size-524288            0      0 524288    1  128 : tunables    1    1    0 : slabdata      0      0      0
size-262144(DMA)       0      0 262144    1   64 : tunables    1    1    0 : slabdata      0      0      0
size-262144            0      0 262144    1   64 : tunables    1    1    0 : slabdata      0      0      0
size-131072(DMA)       0      0 131072    1   32 : tunables    8    4    0 : slabdata      0      0      0
size-131072            0      0 131072    1   32 : tunables    8    4    0 : slabdata      0      0      0
size-65536(DMA)        0      0  65536    1   16 : tunables    8    4    0 : slabdata      0      0      0
size-65536             0      0  65536    1   16 : tunables    8    4    0 : slabdata      0      0      0
size-32768(DMA)        0      0  32768    1    8 : tunables    8    4    0 : slabdata      0      0      0
size-32768             4      4  32768    1    8 : tunables    8    4    0 : slabdata      4      4      0
size-16384(DMA)        0      0  16384    1    4 : tunables    8    4    0 : slabdata      0      0      0
size-16384            15     15  16384    1    4 : tunables    8    4    0 : slabdata     15     15      0
size-8192(DMA)         0      0   8192    1    2 : tunables    8    4    0 : slabdata      0      0      0
size-8192             13     13   8192    1    2 : tunables    8    4    0 : slabdata     13     13      0
size-4096(DMA)         0      0   4096    1    1 : tunables   24   12    8 : slabdata      0      0      0
size-4096            221    221   4096    1    1 : tunables   24   12    8 : slabdata    221    221      0
size-2048(DMA)         0      0   2048    2    1 : tunables   24   12    8 : slabdata      0      0      0
size-2048            512    530   2048    2    1 : tunables   24   12    8 : slabdata    265    265      0
size-1024(DMA)         0      0   1024    4    1 : tunables   54   27    8 : slabdata      0      0      0
size-1024           1458   1472   1024    4    1 : tunables   54   27    8 : slabdata    367    368      0
size-512(DMA)          0      0    512    8    1 : tunables   54   27    8 : slabdata      0      0      0
size-512             542    592    512    8    1 : tunables   54   27    8 : slabdata     74     74      0
size-256(DMA)          0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
size-256             888   1005    256   15    1 : tunables  120   60    8 : slabdata     67     67      0
size-128(DMA)          0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
size-64(DMA)           0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
size-64             2470   5133     64   59    1 : tunables  120   60    8 : slabdata     87     87      0
size-32(DMA)           0      0     32  112    1 : tunables  120   60    8 : slabdata      0      0      0
size-128            1385   1560    128   30    1 : tunables  120   60    8 : slabdata     52     52      0
size-32             7730   8064     32  112    1 : tunables  120   60    8 : slabdata     72     72      0
kmem_cache           147    160    384   10    1 : tunables   54   27    8 : slabdata     16     16      0

[-- Attachment #4: Type: TEXT/plain, Size: 2824 bytes --]

[ 4863.335825] SysRq : Show Memory
[ 4863.335949] Mem-info:
[ 4863.335952] DMA per-cpu:
[ 4863.335954] CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[ 4863.336008] CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[ 4863.336010] CPU    2: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[ 4863.336012] CPU    3: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[ 4863.336014] CPU    4: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[ 4863.336016] CPU    5: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[ 4863.336018] CPU    6: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[ 4863.336019] CPU    7: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[ 4863.336021] DMA32 per-cpu:
[ 4863.336022] CPU    0: Hot: hi:   42, btch:   7 usd:  44   Cold: hi:   14, btch:   3 usd:   4
[ 4863.336024] CPU    1: Hot: hi:   42, btch:   7 usd:  17   Cold: hi:   14, btch:   3 usd:   2
[ 4863.336026] CPU    2: Hot: hi:   42, btch:   7 usd:  10   Cold: hi:   14, btch:   3 usd:   1
[ 4863.336028] CPU    3: Hot: hi:   42, btch:   7 usd:  27   Cold: hi:   14, btch:   3 usd:   3
[ 4863.336030] CPU    4: Hot: hi:   42, btch:   7 usd:  35   Cold: hi:   14, btch:   3 usd:   2
[ 4863.336032] CPU    5: Hot: hi:   42, btch:   7 usd:  20   Cold: hi:   14, btch:   3 usd:   1
[ 4863.336034] CPU    6: Hot: hi:   42, btch:   7 usd:  36   Cold: hi:   14, btch:   3 usd:   2
[ 4863.336035] CPU    7: Hot: hi:   42, btch:   7 usd:  36   Cold: hi:   14, btch:   3 usd:   3
[ 4863.336038] Active:8805 inactive:7364 dirty:6 writeback:0 unstable:0
[ 4863.336039]  free:20589 slab:7401 mapped:1133 pagetables:247 bounce:0
[ 4863.336041] DMA free:10764kB min:100kB low:124kB high:148kB active:344kB inactive:184kB present:11292kB pages_scanned:0 all_unreclaimable? no
[ 4863.336043] lowmem_reserve[]: 0 173 173 173
[ 4863.336046] DMA32 free:71592kB min:1632kB low:2040kB high:2448kB active:34876kB inactive:29272kB present:177760kB pages_scanned:0 all_unreclaimable? no
[ 4863.336048] lowmem_reserve[]: 0 0 0 0
[ 4863.336050] DMA: 211*4kB 194*8kB 147*16kB 88*32kB 34*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 10764kB
[ 4863.336056] DMA32: 486*4kB 488*8kB 1373*16kB 842*32kB 235*64kB 10*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 71592kB
[ 4863.336391] Swap cache: add 20537385, delete 20539682, find 2655799/6042712, race 157+192
[ 4863.336393] Free swap  = 3142156kB
[ 4863.336395] Total swap = 3145712kB
[ 4863.336396] Free swap:       3142156kB
[ 4863.337203] 49152 pages of RAM
[ 4863.337206] 2036 reserved pages
[ 4863.337209] 11197 pages shared
[ 4863.337210] 377 pages swap cached

^ permalink raw reply	[flat|nested] 8+ messages in thread

* RE: arcmsr & areca-1660 - strange behaviour under heavy load
  2008-02-26  9:35   ` Nikola Ciprich
@ 2008-02-26 10:30     ` nickcheng
  2008-02-26 17:43     ` Andrew Morton
  1 sibling, 0 replies; 8+ messages in thread
From: nickcheng @ 2008-02-26 10:30 UTC (permalink / raw)
  To: 'Nikola Ciprich', 'Andrew Morton'
  Cc: linux-kernel, linux-scsi, 'Erich Chen', kopi, kevin34,
	billion.wu

Hi Nikola,
As I said, we will test on our site.
Our support team will help you to settle the issue.
Sorry for your inconvenience,

-----Original Message-----
From: Nikola Ciprich [mailto:extmaillist@linuxbox.cz] 
Sent: Tuesday, February 26, 2008 5:36 PM
To: Andrew Morton
Cc: linux-kernel@vger.kernel.org; linux-scsi@vger.kernel.org; Nick Cheng;
Erich Chen; kopi@linuxbox.cz
Subject: Re: arcmsr & areca-1660 - strange behaviour under heavy load

Hi

On Sun, 24 Feb 2008, Andrew Morton wrote:

Hi Andrew,
thanks a lot for reply, I'm attaching requested information.
please let me know if You need more information/testing, whatever.
I'll be glad to help.
BR
nik

>> Areca support doesn't seem to be very interested in the problem :-(
>
> (cc's added)
>
> Please get the machine into this state of memory exhaustion then take
> copies of the output of the following, and send them via reply-to-all to
> this email:
>
> - cat /proc/meminfo
>
> - cat /proc/slabinfo
>
> - dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
>
> Thanks.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>

-- 


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: arcmsr & areca-1660 - strange behaviour under heavy load
  2008-02-26  9:35   ` Nikola Ciprich
  2008-02-26 10:30     ` nickcheng
@ 2008-02-26 17:43     ` Andrew Morton
  2008-02-26 19:29       ` Nikola Ciprich
  1 sibling, 1 reply; 8+ messages in thread
From: Andrew Morton @ 2008-02-26 17:43 UTC (permalink / raw)
  To: Nikola Ciprich; +Cc: linux-kernel, linux-scsi, Nick Cheng, Erich Chen, kopi

On Tue, 26 Feb 2008 10:35:31 +0100 (CET) Nikola Ciprich <extmaillist@linuxbox.cz> wrote:

> Hi
> 
> On Sun, 24 Feb 2008, Andrew Morton wrote:
> 
> Hi Andrew,
> thanks a lot for reply, I'm attaching requested information.
> please let me know if You need more information/testing, whatever.
> I'll be glad to help.
> BR
> nik
> 
> >> Areca support doesn't seem to be very interested in the problem :-(
> >
> > (cc's added)
> >
> > Please get the machine into this state of memory exhaustion then take
> > copies of the output of the following, and send them via reply-to-all to
> > this email:
> >
> > - cat /proc/meminfo
> >
> > - cat /proc/slabinfo
> >
> > - dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
> >
> > Thanks.

Alas, that all looks OK to me.

You never get any out-of-memory messages, and no oom-killing messages?

Possibly what is happening here is that in this low-memory condition, some
of the driver's internal memory-allocation attempts are failing, and the
driver isn't correctly handling this.  This is a rare situation which may
well not have been hit in anyone else's testing.

I expect that the Areca engineers will be able to reproduce this with a
suitably small "mem=" kernel boot option.  If not, they could perhaps
investigate the kernel's fault-injection framework, which permits
simulation of page allocation failures.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: arcmsr & areca-1660 - strange behaviour under heavy load
  2008-02-26 17:43     ` Andrew Morton
@ 2008-02-26 19:29       ` Nikola Ciprich
  2008-02-26 21:04         ` Zan Lynx
  0 siblings, 1 reply; 8+ messages in thread
From: Nikola Ciprich @ 2008-02-26 19:29 UTC (permalink / raw)
  To: Andrew Morton; +Cc: linux-kernel, linux-scsi, Nick Cheng, Erich Chen, kopi

Hi Andrew,
no, right now I have the machine in the weird state, swap is empty (3GB), 
and so is bigger part of RAM (~100MB free), and the gcc crashes even when 
trying to compile c program with empty main function. so it doesn't seem 
to be problem with memory exhaustion.
Hopefully the areca guys will be able to find out what is going on. But 
anyways, if You'll have any other idea what should I check/try, please let 
me know, as I have to admit that I'd really like to hunt it down myself 
(and yes, there is some vanity on my side here :))
thanks a lot once more
cheers
nik



  On Tue, 26 Feb 2008, 
Andrew Morton wrote:

> On Tue, 26 Feb 2008 10:35:31 +0100 (CET) Nikola Ciprich <extmaillist@linuxbox.cz> wrote:
>
>> Hi
>>
>> On Sun, 24 Feb 2008, Andrew Morton wrote:
>>
>> Hi Andrew,
>> thanks a lot for reply, I'm attaching requested information.
>> please let me know if You need more information/testing, whatever.
>> I'll be glad to help.
>> BR
>> nik
>>
>>>> Areca support doesn't seem to be very interested in the problem :-(
>>>
>>> (cc's added)
>>>
>>> Please get the machine into this state of memory exhaustion then take
>>> copies of the output of the following, and send them via reply-to-all to
>>> this email:
>>>
>>> - cat /proc/meminfo
>>>
>>> - cat /proc/slabinfo
>>>
>>> - dmesg -c > /dev/null ; echo m > /proc/sysrq-trigger ; dmesg -c
>>>
>>> Thanks.
>
> Alas, that all looks OK to me.
>
> You never get any out-of-memory messages, and no oom-killing messages?
>
> Possibly what is happening here is that in this low-memory condition, some
> of the driver's internal memory-allocation attempts are failing, and the
> driver isn't correctly handling this.  This is a rare situation which may
> well not have been hit in anyone else's testing.
>
> I expect that the Areca engineers will be able to reproduce this with a
> suitably small "mem=" kernel boot option.  If not, they could perhaps
> investigate the kernel's fault-injection framework, which permits
> simulation of page allocation failures.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>

-- 


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: arcmsr & areca-1660 - strange behaviour under heavy load
  2008-02-26 19:29       ` Nikola Ciprich
@ 2008-02-26 21:04         ` Zan Lynx
  2008-02-27  1:53           ` nickcheng
  0 siblings, 1 reply; 8+ messages in thread
From: Zan Lynx @ 2008-02-26 21:04 UTC (permalink / raw)
  To: Nikola Ciprich
  Cc: Andrew Morton, linux-kernel, linux-scsi, Nick Cheng, Erich Chen,
	kopi

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]


On Tue, 2008-02-26 at 20:29 +0100, Nikola Ciprich wrote:
> Hi Andrew,
> no, right now I have the machine in the weird state, swap is empty (3GB), 
> and so is bigger part of RAM (~100MB free), and the gcc crashes even when 
> trying to compile c program with empty main function. so it doesn't seem 
> to be problem with memory exhaustion.

Maybe memory fragmentation?  Perhaps the driver tries to allocate a
large block of memory and cannot find a continuous block of the right
size.

Maybe the driver developers used different kernel .config options than
you are using.  

Try increasing the value in /proc/sys/vm/min_free_kbytes.

Try switching some things like SLAB or SLUB, try booting with
kernelcore=512M to enable the Movable memory zone, or try 64-bit vs
32-bit kernels. 
-- 
Zan Lynx <zlynx@acm.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* RE: arcmsr & areca-1660 - strange behaviour under heavy load
  2008-02-26 21:04         ` Zan Lynx
@ 2008-02-27  1:53           ` nickcheng
  0 siblings, 0 replies; 8+ messages in thread
From: nickcheng @ 2008-02-27  1:53 UTC (permalink / raw)
  To: 'Nikola Ciprich'
  Cc: 'Andrew Morton', linux-kernel, linux-scsi,
	'Erich Chen', kopi, support, 'Zan Lynx'

Hi Nikola,
Please put support@areca.com.tw in the loop.
I am sure Areca support, Kevin, has taken over your case.
If you like, please let him know your configuration and operations to
synchronize both sides.
Thank you for your patience and sorry for your inconvenience,

-----Original Message-----
From: Zan Lynx [mailto:zlynx@acm.org] 
Sent: Wednesday, February 27, 2008 5:04 AM
To: Nikola Ciprich
Cc: Andrew Morton; linux-kernel@vger.kernel.org; linux-scsi@vger.kernel.org;
Nick Cheng; Erich Chen; kopi@linuxbox.cz
Subject: Re: arcmsr & areca-1660 - strange behaviour under heavy load

On Tue, 2008-02-26 at 20:29 +0100, Nikola Ciprich wrote:
> Hi Andrew,
> no, right now I have the machine in the weird state, swap is empty (3GB), 
> and so is bigger part of RAM (~100MB free), and the gcc crashes even when 
> trying to compile c program with empty main function. so it doesn't seem 
> to be problem with memory exhaustion.

Maybe memory fragmentation?  Perhaps the driver tries to allocate a
large block of memory and cannot find a continuous block of the right
size.

Maybe the driver developers used different kernel .config options than
you are using.  

Try increasing the value in /proc/sys/vm/min_free_kbytes.

Try switching some things like SLAB or SLUB, try booting with
kernelcore=512M to enable the Movable memory zone, or try 64-bit vs
32-bit kernels. 
-- 
Zan Lynx <zlynx@acm.org>

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2008-02-27  1:53 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-02-23 11:20 arcmsr & areca-1660 - strange behaviour under heavy load Nikola Ciprich
2008-02-25  0:10 ` Andrew Morton
2008-02-26  9:35   ` Nikola Ciprich
2008-02-26 10:30     ` nickcheng
2008-02-26 17:43     ` Andrew Morton
2008-02-26 19:29       ` Nikola Ciprich
2008-02-26 21:04         ` Zan Lynx
2008-02-27  1:53           ` nickcheng

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox