Re: direct_access, pinning and truncation

[Dave Chinner <david@fromorbit.com>]

On Sun, Oct 19, 2014 at 02:08:07PM +0300, Boaz Harrosh wrote:
> On 10/10/2014 05:24 PM, Matthew Wilcox wrote:
> <>
> > 
> > I'm assuming that we come up with *some* way to solve the missing struct
> > page problem.  Whether it's restructuring splice, O_DIRECT and RDMA to do
> > without struct pages, 
> 
> That makes no sense to me, where will it end? You are doubling the size of the
> code to have two paths, and there will always be a subsystem you did not touch
> and is missing support. And why? page was already invented to do exactly what you
> want, track state of a PFN.
.....
> > whether it's coming up
> > with some other data structure that takes the place of struct page for
> > DAX ... 
> 
> Again. Why reinvent the wheel when the old one works perfectly and does
> everything you want, including the most important aspect. Not adding any
> new infrastructure, and/or modifying any code. So why even think about it?
> 
> > doesn't matter for this part of the conversation.
> > 
> 
> I agree, this does not solve the reference problem, in this case DAX will
> need an new entry into the FS to communicate delayed free-block. But as Jan
> pointed out this is not against current FS structure.
> 
> I think lots of current DAX problems and performance short comings can be
> solved very nicely if we assume we have struct-page for pmem. For example
> the use of the page-lock instead of the i_mutex we take today.

Which makes me look at what DAX is intended for.

DAX is an enabler, allowing us to get direct access to PMEM with
*existing filesystem technologies*.  I don't want to have to add new
extent management functions to XFS to add temporary references to
allow DAX to hold onto extents after an inode has been freed because
some RDMA app has pinned the PMEM and forgot to let it go. That way
lies madness for existing filesystems - yes, we can add such warts
to them, but it's ugly, nasty and needed only by a very, very small
lunatic fringe of users.

IMO, this proposal is way outside the original DAX-replaces-XIP scope;
I really don't think that requiring extensive modifications to
filesystems to use DAX is a good idea. Apart from it being contrary to the
original architectural goal of DAX (which was "enable direct access
with minimal filesystem implementation impact"), we risk significant
impact on non-DAX users by requiring architectural changes to the
underlying filesystems to support DAX.

So my question is this: at what point do we say "out of scope for
DAX, make this work with a native PMEM filesystem"?  DAX as it
stands fills the "95% of what people need" goal with minimal effort;
our efforts should be focussed on merging what we have, not creeping
the scope and making it harder to implement and get merged.

If we want RDMA into PMEM devices or direct IO to/from persisten
memory, then I'd suggest that this is functionality that belongs in
native PMEM storage devices/filesystems and should be designed to be
efficient in that environment way from the ground up.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com