Virtual memory problem with kernel-2.4.18.xsmp

["yun lin" <liny.linux@gmail.com>]

Hi all,
    I am using redhat 8 (kernel 2.4.18 xsmp) on an old cluster(2CPU
each nodes, 2GB RAM, 4GB swap and NFS). Recently I encounted a problem
of virtual memory. A parallel progrom terminated with Segementation
fault (Signal 11). While using gdb to debug, it turns out that the
exact error message is "virtual memory exhausted".  The program was
compiled using Intel compiler 8.1. I had compiled it on a CentOS
5.2(2GB RAM, 4GB swap) with Intel compiler 10.1, and the program runs
well. It requires about 200MB memory, as watching by 'top' command.
And during its running, 'kswapd' demon would be awaken.
    While running this program in the cluster, it always dies when the
RSS reach about 150MB with a signal 11 (or using gdb with 'virtual
memory exhaust'). During its running, the system have more than 1.2GB
free RAM 4GB free swap and the 'kswapd' keeps in 'SW' status. I had
check the /etc/security/limits.conf file, which is a empty file(no
limits setted). And I had try bash and csh, with the stack size,
virtual memory, max memory size, file size unlimited. I had also
double the max_map_count in /proc/sys/vm/. But all of these have not
effect on the program, it still dies when it accupied about 150MB RAM
and report sig 11 (in gdb 'VM exhauxted'). I had manual set some
limits on the virtual memory (such as 'virtual memory' limit in the
shell or max_map_count), the program dies with some different message
like 'forrtl: not enough virtual memory ...."
    Are there any other limits that restrain a process from getting
more virtual memory? I try to speed up the fresh frequency of
'kswapd', but I can't file 'freepages' file in my /proc/sys/vm
directory. It is said that this keys had been removed since kernel
2.4, so how can I increase the fresh frequency of 'kswapd'? Besides,
are there any special rules in virtual memory control of SMP kernel?
    Thanks for reading my letters, hope to get some helps or clues.

Lin Yun
2008.9.17