Re: why swap at all?

[Nick Piggin <nickpiggin@yahoo.com.au>]

Buddy Lumpkin wrote:
>>>Couple that with the fact that there are many pte's pointing at the same
>>>physical page (shared page) in many cases where many processes 
>>>
>>>are running
>>>on the system. Because all of the references to that page must be removed
>>>before the page can be evicted, there are some absolute 
>>>limitations in the
>>>rate that pages can be evicted from memory as the number of processes
>>>running on the system and the total amount of memory increases.
>>>
> 
> 
>>This is still many orders of magnitude faster than filling the page
>>from disk, and you typically don't reclaim much of mapped memory anyway.
> 
> 
> This discussion went broke-minded again. Your still picturing that single
> IDE hard drive in your workstation and im talking about big iron, large
> databases, etc.. where the total amount of aggregate disk I/O is completely
> limited by the rate you can evict pages from the pagecache.
> 
> Picture 6 to 7 fibre channel cards with over 70% utilization during peak
> usage times connected to a large EMC storage array with 64GB of non-volatile
> cache.
> 

I can picture it but I don't know how the kernel is going to handle
it. All I am doing is speaking from what I have seen.

http://marc.theaimsgroup.com/?l=linux-kernel&m=107817776322044&w=2

This post for example, has profiles of a 32 CPU system with 16 FC
controllers and over 1000 disks, doing some database workload. Does
this qualify as big iron?

In the bottom profile, you see the disks being kept busy with 50%
idle time. The top 6 functions are all to do with generating IO
requests and pushing them through the block layer, none of them
involve memory reclaim.

There are profiles from a different setup in a related thread here:

http://groups.google.com.au/groups?q=g:thl3816668183d&dq=&hl=en&lr=&ie=UTF-8&selm=1yjKu-7qU-1%40gated-at.bofh.it&rnum=9

I think we see kmem_cache_alloc make a miserable showing for the
memory allocation team, but it wouldn't even be there if the
profile were sorted by ticks (the left hand column).


Now If you had some experiences of memory reclaim slowing down
block IO, I'd love to hear them because that is related to an area
that I am looking at currently. I'm not saying what you claim is
impossible, but it is something that shouldn't happen and we don't
relly see... You're continuing to insist there is a problem but
that simply isn't helpful without further details.