Re: [PATCH] [RFC] fadvise: Add _VOLATILE,_ISVOLATILE, and _NONVOLATILE flags

[Rik van Riel <riel@redhat.com>]

On 11/22/2011 10:40 AM, Robert Love wrote:

>     3) Isn't it too slow for something like Firefox to
>        create a new tmpfs object for every single throw-away
>        cache object?
>
>
> Nope, although user-space might want to create larger mappings and
> sub-divide them if its objects are *really* small.

At first glance, it sounds like a great idea to have a program
like Firefox create a new object for every tab.

However, this quickly runs into the practical problem that
Firefox does not know in advance how much memory each tab will
require.

Without a good way to size each mapping in advance, it may not
be practical to group related bits of memory together.

Never mind that it would require userspace programs to grow new
memory allocators...

>     Johannes, Jon and I have looked at an alternative way to
>     allow the kernel and userspace to cooperate in throwing
>     out cached data.  This alternative way does not touch
>     the alloc/free fast path at all, but does require some
>     cooperation at "shrink cache" time.
>
>
> I'm open to whatever works, and I need to think about your proposal
> more, but it sounds inferior to the model John is proposing. I think we
> can sum the two models up as,
>
> - User-space pins, unpins objects, and kernel handles eviction
> - Kernel lets user-space register callbacks for notification during
> cache shrink, user handles eviction
>
> Right?

Yes, this characterization is correct.

Note how in the object I proposed, when there is no memory
pressure, userspace can hold on to freed pages and quickly
reuse them for something else.

There is no need for pages to be handed back to the kernel,
and then zeroed again when they get handed back to userspace
later on.

There is also no need to write a new memory allocator for
programs that want to pack related objects together in one
_VOLATILE mapping.

The reason is that, on memory pressure, any arbitrary set of
pages can be nuked from userspace, so any memory allocator will
be compatible.

> Actually, I can see the need for both. But the first is simpler, lets
> the kernel evict pages from user-space even in low memory situations
> (since you can call the cache shrinker atomically), and relies on the
> fact that the "free caches" action is just jettisoning the pages, so the
> kernel can do it. The actual work is in repopulating the caches.
>
> That last point, btw, is why user-space developers like ashmem: It fits
> their models of how they do work. They pin/unpin objects as part of
> their normal access and repopulating is tied into the re-pin path, which
> is where it makes sense.

Which userspace programs work that way already?