Re: [PATCH 0/2] vmalloc: Introduce vmap_file()

Re: [PATCH 0/2] vmalloc: Introduce vmap_file()

[Andrew Morton]

On Thu, 30 Jan 2025 16:18:04 -0800 "Vishal Moola (Oracle)" <vishal.moola@gmail.com> wrote:

> Currently, users have to call vmap() or vmap_pfn() to map pages to
> kernel virtual space. vmap() requires the page references, and
> vmap_pfn() requires page pfns. If we have a file but no page references,
> we have to do extra work to map them.
> 
> Create a function, vmap_file(), to map a specified range of a given
> file to kernel virtual space. Also convert a user that benefits from
> vmap_file().
> 

Seems like a pretty specialized thing.  Have you identified any other
potential users of vmap_file()?  I couldn't see any.

If drm is likely to remain the only user of this, perhaps we should
leave the code down in drivers/gpu/drm for now?


Also, the amount of copy-n-pasting from vmap() into vmap_file() is
undesirable - code size, maintenance overhead, etc.

Re: [PATCH 1/2] mm/vmalloc: Introduce vmap_file()

[Christoph Hellwig]

On Thu, Jan 30, 2025 at 04:18:05PM -0800, Vishal Moola (Oracle) wrote:
> +	rcu_read_lock();
> +	xas_for_each(&xas, folio, last) {

This only maps folios currently in the page cache, which makes it
usefull for everything except ramfs-style purely in-memory file systems.
I.e. for the shmem use case in the second patch it fails to swap in
swapped out tmpfs folios.

> +EXPORT_SYMBOL(vmap_file);

EXPORT_SYMBOL_GPL for any advances vmalloc-layer functionality, please.

Re: [PATCH 0/2] vmalloc: Introduce vmap_file()

[Christoph Hellwig]

On Thu, Jan 30, 2025 at 04:18:04PM -0800, Vishal Moola (Oracle) wrote:
> Currently, users have to call vmap() or vmap_pfn() to map pages to
> kernel virtual space. vmap() requires the page references, and
> vmap_pfn() requires page pfns. If we have a file but no page references,
> we have to do extra work to map them.
> 
> Create a function, vmap_file(), to map a specified range of a given
> file to kernel virtual space. Also convert a user that benefits from
> vmap_file().

As far as I can tell there is exatly one user that maps file pages
into vmalloc space.  It's a pretty odd thing to do, so figuring out
a way to get rid of that might be a better use of time.

Re: [PATCH 0/2] vmalloc: Introduce vmap_file()

[Vishal Moola]

On Thu, Jan 30, 2025 at 4:48 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Thu, 30 Jan 2025 16:18:04 -0800 "Vishal Moola (Oracle)" <vishal.moola@gmail.com> wrote:
>
> > Currently, users have to call vmap() or vmap_pfn() to map pages to
> > kernel virtual space. vmap() requires the page references, and
> > vmap_pfn() requires page pfns. If we have a file but no page references,
> > we have to do extra work to map them.
> >
> > Create a function, vmap_file(), to map a specified range of a given
> > file to kernel virtual space. Also convert a user that benefits from
> > vmap_file().
> >
>
> Seems like a pretty specialized thing.  Have you identified any other
> potential users of vmap_file()?  I couldn't see any.
>
> If drm is likely to remain the only user of this, perhaps we should
> leave the code down in drivers/gpu/drm for now?

This function is generally useful for file-systems that use the pagecache.
I simply chose to highlight the most obvious user that benefits from it (and
so that the function is introduced with a user).

I haven't identified any other specific users of vmap_file() myself. I know
Matthew has some other ideas for it; I've cc-ed him so he can chime in.

>
> Also, the amount of copy-n-pasting from vmap() into vmap_file() is
> undesirable - code size, maintenance overhead, etc.

I wasn't particularly a fan of it either, but I couldn't find a more readable
way to do this (without reorganizing multiple other functions). Aside from
the initial flags checks, the rest of the function is slightly different from
vmap(), so calling existing functions won't suffice.

I considered passing more arguments through to vmap(), but I think that
would make the code more confusing, especially because the 2 functions
have some different usage prerequisites.

Re: [PATCH 1/2] mm/vmalloc: Introduce vmap_file()

[Vishal Moola]

On Thu, Jan 30, 2025 at 11:09 PM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Thu, Jan 30, 2025 at 04:18:05PM -0800, Vishal Moola (Oracle) wrote:
> > +     rcu_read_lock();
> > +     xas_for_each(&xas, folio, last) {
>
> This only maps folios currently in the page cache, which makes it
> usefull for everything except ramfs-style purely in-memory file systems.
> I.e. for the shmem use case in the second patch it fails to swap in
> swapped out tmpfs folios.

Ah, I see. I can drop that patch then. Its primary purpose was to provide a
user for vmap_file(). As you've pointed out, that won't work with tmpfs or
anon pages. I'll hold off on a v2 until there are better usecases for
vmap_file().

> > +EXPORT_SYMBOL(vmap_file);
>
> EXPORT_SYMBOL_GPL for any advances vmalloc-layer functionality, please.

Ok, I'll keep that in mind in the future.

Re: [PATCH 0/2] vmalloc: Introduce vmap_file()

[Brendan Jackman]

On Mon Feb 3, 2025 at 6:53 PM UTC, Vishal Moola wrote:
> On Thu, Jan 30, 2025 at 4:48 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>>
>> On Thu, 30 Jan 2025 16:18:04 -0800 "Vishal Moola (Oracle)" <vishal.moola@gmail.com> wrote:
>>
>> > Currently, users have to call vmap() or vmap_pfn() to map pages to
>> > kernel virtual space. vmap() requires the page references, and
>> > vmap_pfn() requires page pfns. If we have a file but no page references,
>> > we have to do extra work to map them.
>> >
>> > Create a function, vmap_file(), to map a specified range of a given
>> > file to kernel virtual space. Also convert a user that benefits from
>> > vmap_file().
>> >
>>
>> Seems like a pretty specialized thing.  Have you identified any other
>> potential users of vmap_file()?  I couldn't see any.
>>
>> If drm is likely to remain the only user of this, perhaps we should
>> leave the code down in drivers/gpu/drm for now?
>
> This function is generally useful for file-systems that use the pagecache.
> I simply chose to highlight the most obvious user that benefits from it (and
> so that the function is introduced with a user).
>
> I haven't identified any other specific users of vmap_file() myself. I know
> Matthew has some other ideas for it; I've cc-ed him so he can chime in.

Not much to add but just to confirm - yep, this seems like it might be
useful as a part of the solution to the page cache perf issue[1] with
ASI that I spoke about (briefly and chaotically) at the end of the
LSF/MM/BPF session[0] on ASI this year.

[0] https://lwn.net/Articles/1016013/
[1] https://lore.kernel.org/linux-mm/20250129144320.2675822-1-jackmanb@google.com/

But, for the moment this is all still pretty vague stuff, not at all
clear yet that this idea makes total sense. Hopefully I'll be able to
follow up in a few weeks after I've made some time to stare at/prototype
things.