Re: cache limit

["Joseph Malicki" <jmalicki@starbak.net>]

I've had experience with unneeded, *mapped* pages that would be ideally
flushed oppressing needed mapped and unmapped pages.
Test case: grep --mmap SOME_STRING_I_WONT_FIND some_multi-GB-file

Sure, it's bad programming etc, but in that case, once those pages are
mapped, they can't be forcibly unmapped even though
in a utopian VM they would be discarded as unneeded.

This could very well be the problem?

-joe

----- Original Message ----- 
From: "William Lee Irwin III" <wli@holomorphy.com>
To: "Takao Indoh" <indou.takao@soft.fujitsu.com>
Cc: "Mike Fedyk" <mfedyk@matchmail.com>; <linux-kernel@vger.kernel.org>
Sent: Wednesday, August 27, 2003 5:45 AM
Subject: Re: cache limit


> On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote:
> > This problem happened a few month ago and the detailed data does not
> > remain. Therefore it is difficult to know what is essential cause for
> > this problem, but, I guessed that pagecache used as I/O cache grew
> > gradually during system running, and finally it oppressed memory.
>
> But this doesn't make any sense; the only memory you could "oppress"
> is pagecache.
>
>
> On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote:
> > Besides this problem, there are many cases where increase of pagecache
> > causes trouble, I think.
> > For example, DBMS.
> > DBMS caches index of DB in their process space.
> > This index cache conflicts with the pagecache used by other
applications,
> > and index cache may be paged out. It cause uneven response of DBMS.
> > In this case, limiting pagecache is effective.
>
> Why is it effective? You're describing pagecache vs. pagecache
> competition and the DBMS outcompeting the cooperating applications for
> memory to the detriment of the workload; this is a very different
> scenario from what "limiting pagecache" sounds like.
>
> How do you know it would be effective? Have you written a patch to
> limit it in some way and tried running it?
>
>
> On Tue, 26 Aug 2003 02:46:34 -0700, William Lee Irwin III wrote:
> >> One thing I thought of after the post was whether they actually had in
> >> mind tunable hard limits on _unmapped_ pagecache, which is, in fact,
> >> useful. OTOH that's largely speculation and we really need them to
> >> articulate the true nature of their problem.
>
> On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote:
> > I also think that is effective. Empirically, in the case where pagecache
> > causes memory shortage, most of pagecache is unmapped page. Of course
> > real problem may not be pagecashe, as you or Mike said.
>
> How do you know most of it is unmapped?
>
> At any rate, the above assigns a meaningful definition to the words you
> used; it does not necessarily have anything to do with the issue you're
> trying to describe. If you could start from the very basics, reproduce
> the problem, instrument the workload with top(1) and vmstat(1), and find
> some way to describe how the performance is inadequate (e.g. performance
> metrics for your running DBMS/whatever in MB/s or transactions/s etc.),
> it would be much more helpful than proposing a solution up front.
> Without any evidence, we can't know it is a solution at all, or that
> it's the right solution.
>
>
> -- wli
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>