Re: [PATCH 3/5] block: Add support for DAX on block devices

[Matthew Wilcox <willy@linux.intel.com>]

On Tue, Jun 30, 2015 at 04:19:49AM -0700, Christoph Hellwig wrote:
> On Mon, Jun 29, 2015 at 04:02:30PM -0400, Matthew Wilcox wrote:
> > From: Matthew Wilcox <willy@linux.intel.com>
> > 
> > Without this patch, accesses to a file on a filesystem on a block device
> > could be done without the page cache, but accessing the block device
> > itself would always go through the page cache.
> > 
> > Now reads and writes to a block device that is capable of DAX will always
> > bypass the page cache.  Loads and stores to an mmapped block device will
> > bypass the page cache if the user specified O_DIRECT.  This opt-in from
> > the user is necessary because DAX mappings are currently incompatible
> > with RDMA and O_DIRECT I/Os with non-DAX files.
> 
> Using O_DIRECT for this seems like a pretty horrible hack, so I'd like
> to see a really good justification of using this over other interfaces.

O_DIRECT means "bypass the page cache", which is what this does (now it's
able to apply to mmap too).

> Also it needs a Cc to linux-api and an entry in the open man page, and
> and even better explanation of why we only support this interface on
> block devices but not file systems.

Um, we do support this for filesystems with DAX.  The inconsistency we
have is that if you have a direct-access-capable block device, currently
files in a filesystem on it get the bypass-page-cache treatment, but if
you use the raw block device directly, that mapping doesn't.

> Last but least I supect we'll need a runtime option for direct_access
> support in the brd devices, as we're now going to use the regular
> block device path less and less.

I'm getting there; I was working on getting DAX to dynamically map the
pages that it used (rather than relying on them being permanently part
of the direct mapping), but I had to set that work aside temporarily.
That lets us just delete the compile option, and have direct_access
always work on brd.