Re: [PATCH 01 of 16] remove nr_scan_inactive/active

Rik van Riel
2007-06-28T23:16:45Z

Andrew Morton wrote:
> On Thu, 28 Jun 2007 19:04:05 -0400
> Rik van Riel <riel@redhat.com> wrote:
>
>>> Sigh. We had a workload (forget which, still unfixed) in which things
>>> would basically melt down in that linear anon_vma walk, walking 10,000 or
>>> more vma's. I wonder if that's what's happening here?
>> That would be a large multi-threaded application that fills up
>> memory. Customers are reproducing this with JVMs on some very
>> large systems.
>
> So.... does that mean "yes, it's scanning a lot of vmas"?

Not necessarily.

The problem can also be reproduced if you have many
threads, from "enough" CPUs, all scanning pages in
the same huge VMA.

> If so, I expect there will still be failure modes, whatever we do outside
> of this. A locked, linear walk of a list whose length is
> application-controlled is going to be a problem. Could be that we'll need
> an O(n) -> O(log(n)) conversion, which will be tricky in there.

Scanning fewer pages in the pageout path is probably
the way to go.

No matter how efficient we make the scanning of one
individual page, we simply cannot scan through 1TB
worth of anonymous pages (which are all referenced
because they've been there for a week) in order to
deactivate something.

Systems that big are only a year or two away from
general purpose use.

--
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is. Each group
calls the other unpatriotic.