Re: [-mm PATCH v2 00/25] get_user_pages() for dax pte and pmd mappings

[Jeff Moyer <jmoyer@redhat.com>]

Dan Williams <dan.j.williams@intel.com> writes:

> On Thu, Dec 10, 2015 at 10:08 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>> Dan Williams <dan.j.williams@intel.com> writes:
>>
>>> Summary:
>>>
>>> To date, we have implemented two I/O usage models for persistent memory,
>>> PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
>>> userspace).  This series adds a third, DAX-GUP, that allows DAX mappings
>>> to be the target of direct-i/o.  It allows userspace to coordinate
>>> DMA/RDMA from/to persistent memory.
>>>
>>> The implementation leverages the ZONE_DEVICE mm-zone that went into
>>> 4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
>>> and dynamically mapped by a device driver.  The pmem driver, after
>>> mapping a persistent memory range into the system memmap via
>>> devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
>>> page-backed pmem-pfns via flags in the new pfn_t type.
>>
>> So, this basically means that an admin has to decide whether or not DMA
>> will be used on a given device before making a file system on it.  That
>> seems like an odd requirement.  There's also a configuration option of
>> whether to put those backing struct pages into DRAM or PMEM (which, of
>> course, will be dictated by the size of pmem).  I really think we should
>> reconsider this approach.
>>
>> First, the admin shouldn't have to choose whether or not DMA will be
>> done on the file system.
>
> To be clear it's not "whether or not DMA will be done on the file
> system", it's whether or not both DMA and DAX will be done
> simultaneously on the filesystem.

Fair point, but I'd view one of those configurations as not recommended.
To be clear, if you're just going to use the device for block based
access, using btt is the safer option.

> DAX is already a capability that an admin can inadvertently disable by
> mis-configuring the alignment of a partition [1].

Heh, using my own commit against me? ;-) Anyway, the commit message
suggests that dax *could* be supported on misaligned partitions.

> Why not also disable it when DMA support is not configured and force
> the fs back to page-cache?  Namespace creation tooling in userspace
> can default to enabling DAX + DMA.

Well, the only reason I can come up with is manufactured:  we've forced
the admin to decide between having that extra space for storage and
doing DMA, and he or she opted for more space.

>> Second, eating up storage space to track
>> mostly unused struct pages seems like a waste.  Is there no future for
>> the "introduce __pfn_t, evacuate struct page from sgls"[1] approach?
>> And if not, is there some other way we can solve this problem?
>
> I'm still very much interested in revisiting the page-less mechanisms
> over time, but given comments like Dave's [2], it's not on any short
> term horizon.

OK.

>> I know dynamic allocation of struct pages is scary, but is it more tractable
>> than no pages for DMA?
>
> I wasn't convinced that it would be any better given the need to
> allocate at section granularity at fault time.  It would still require
> ZONE_DEVICE or something similar.  Waiting until get_user_pages() time
> to allocate pages means we don't get __get_user_pages_fast support.
> It also was not clear to that it would prevent exhaustion of DRAM for
> long-standing / large mappings.

Hmm, yeah, this does seem like a less attractive approach.  Thanks for
enumerating the issues.

-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>