Re: [PATCH v2 7/7] pmem, dax: have direct_access use __pmem annotation

[Ross Zwisler <ross.zwisler@linux.intel.com>]

On Sat, 2015-08-15 at 08:44 -0700, Dan Williams wrote:
> On Sat, Aug 15, 2015 at 2:19 AM, Christoph Hellwig <hch@lst.de> wrote:
> > On Thu, Aug 13, 2015 at 10:51:11AM -0600, Ross Zwisler wrote:
> >> Update the annotation for the kaddr pointer returned by direct_access()
> >> so that it is a __pmem pointer.  This is consistent with the PMEM driver
> >> and with how this direct_access() pointer is used in the DAX code.
> >
> > IFF we stick to the __pmem annotations this looks good.
> >
> > That beeing said I start to really dislike them.  We don't special
> > accesors to read/write from pmem, we just need to explicitly commit
> > it if we want to make it persistent.  So I really don't see the need
> > to treat it special and require all the force casts to and from the
> > attribute.
> 
> I'm not going to put up much of a fight if it's really getting in the way....
> 
> That said, while we don't need special accessors we do need guarantees
> that anything that has written to a persistent memory address has done
> so in a way that wmb_pmem() is able to flush it.  It's more of a "I've
> audited this code path for wmb_pmem() compatibility so use this api to
> write to pmem."
> 
> Perhaps a better way to statically check for missed flushes might be
> to have acquire_pmem_for_write() + release() annotations and the final
> release does a wmb_pmem(), but as far as I can tell the sparse
> acquire/release annotations don't stack.

FWIW I've been on the fence about the __pmem annotations, but my current
thought is that we really do need a way of saying that stores to these
pointers need special care for wmb_pmem() to do its thing and that __pmem does
a reasonably good job of that.  If we can figure out a cooler way, such as the
write() + release() flow Dan is talking about, great.  But I think we need
something to keep us from making errors by storing to PMEM pointers and
leaving data in the processor cache.