limitation in the process address space size,

[Lamia Youseff <lyouseff@cs.ucsb.edu>]

Hi ,
I am performing some experimentation with Xen paravirtualization impact 
on computational workloads, when i observed a strange behavior of the 
domains (dom0 with 256MB, but i did not verify it for domU yet).

As I request more pages to my process address space (through regular 
malloc calls), Xen places a limitation on the process address space size 
in real memory. Above this threshold, the process would swap like crazy, 
and hurt the computational performance of my code.  To be more specific, 
my program pseudo code is shown below. My dom0 is allocated 256 MB at 
initialization.  I then run my code and measure its performance in 
MFLOPS, as well as the swap activity as the process requests more memory.
I observed that when the process resident set size (pages in real 
memory) reaches about 78.39 MB (or 20,070  pages), the process starts 
swapping memory pages. I did not see the same performance degradation 
when i allocate 756MB for dom0. This was not definitely the same 
behavior I get from the native kernel on the same machine (I get no 
performance degradation).
I will appreciate if some one can shed some light on this kernel 
behavior for me. Please ask me if i don't give enough details of the 
problem here.
Thank you,
Lamia Youseff


while (){
    malloc more X bytes;
    fill new bytes  with random numbers;
    do some floating point operations, and measure performance;
    measure swapped pages and RSS (resident set size);
}