Re: userspace pagecache management tool

[Rik van Riel <riel@redhat.com>]

Andrew Morton wrote:

>>> Doing a refault thing would help a bit, but stops working at a certain point.
>> At what point does it stop working?
> 
> We need to store that this-page-got-reclaimed info somewhere.  I don't know
> how space-efficient that is.  Did anyone ever do an implementation?

One 32 bit word per evicted page that we keep track of.

> Of course, the pages need to be re-read again so there's a potential 100%
> hit there, which is in fact not a huge amount in this context.  Depends how
> often it occurs (all the time when refault is being useful?) versus what we
> gain from it.

At this point, when we see that a refaulted page is more
active than the coldest page on the active list, we can
also immediately shrink the active list.  That gives the
next inactive page a better chance to get promoted before
it gets evicted.

>> I am not asking this to be difficult, I just want to get Linux
>> a VM that does not need to be kludged up every time a distro
>> ships it to its customers.
> 
> We have a communication problem here.  Please please please work harder to
> get these problems communicated to the MM developers.  The only vendor MM
> kludge of which I'm aware is a thing which Andrea is working on to address
> a large-shm-segment versus bulk-IO problem (yup, database).
> 
> If you have enough of an understanding of a problem to be able to develop
> and productise a fix then share that info madly, asap.

The problem is that most of the distro patches are
kludges, which we would rather not see again in
future kernels.  They tend to work around the problem,
instead of being a proper fix, since reorganizing the
VM in the middle of a release is not an option.

However, incremental small-to-medium changes might
be an option for the upstream kernel, if you are
interested.

>> I believe one starting point would be a concept that people
>> cannot shoot holes in any more.  That is no guarantee, but
>> as long as the concept has known holes coding it up is likely
>> to be a waste of time since the code will need kludges to
>> deal with the problems later on and we'd be back to square
>> one.
> 
> You mean design it and review the design before coding it?  You'll find few
> objections there.

Few objections, but sadly also very few people interested in
actually reviewing the design :(

If you can find holes in http://linux-mm.org/PageReplacementDesign
please let me know :)

-- 
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.