[PATCH v3 00/15] block, dax updates for 4.4

[PATCH v3 00/15] block, dax updates for 4.4

[Dan Williams]

Changes since v2: [1]

1/ Include a fix to revoke dax-mappings at device teardown time

2/ Include a blkdev_issue_flush implementation for pmem

3/ New block device ioctls to set/query dax mode, make dax opt-in

4/ Collect reviewed-by's

[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-October/002538.html

---

There were several topics developed during this cycle, some are ready
and some are not.  The items needing more review and testing are the
pmem+dax updates with mm, ext4, and xfs dependencies.  Topics like the
dax locking reworks, dax get_user_pages, and write-protecting dax ptes
after fsync/msync.

Instead, this more conservative set represents the fs and mm independent
updates to dax and pmem:

1/ Enable dax mappings for raw block devices

2/ Use blk_queue_{enter|exit} and the availability of 'struct page' to
   fix lifetime issues and races with unloading the pmem driver.

3/ Enable blkdev_issue_flush to fix legacy apps that may dirty pmem via
   dax mmap I/O.  Although legacy environments should consider disabling
   dax for apps that don't expect it.

These depend on the latest 'libnvdimm-for-next' branch from nvdimm.git
and the 'for-4.4/integrity' branch from Jens' tree.  All but the last
three have been out for review previously, and I am looking to submit
them towards the back half of the merge window.

---

Dan Williams (15):
      pmem, dax: clean up clear_pmem()
      dax: increase granularity of dax_clear_blocks() operations
      block, dax: fix lifetime of in-kernel dax mappings with dax_map_atomic()
      libnvdimm, pmem: move request_queue allocation earlier in probe
      libnvdimm, pmem: fix size trim in pmem_direct_access()
      um: kill pfn_t
      kvm: rename pfn_t to kvm_pfn_t
      mm, dax, pmem: introduce pfn_t
      block: notify queue death confirmation
      dax, pmem: introduce zone_device_revoke() and devm_memunmap_pages()
      block: introduce bdev_file_inode()
      block: enable dax for raw block devices
      block, dax: make dax mappings opt-in by default
      dax: dirty extent notification
      pmem: blkdev_issue_flush support


 arch/arm/include/asm/kvm_mmu.h        |    5 -
 arch/arm/kvm/mmu.c                    |   10 +
 arch/arm64/include/asm/kvm_mmu.h      |    3 
 arch/mips/include/asm/kvm_host.h      |    6 -
 arch/mips/kvm/emulate.c               |    2 
 arch/mips/kvm/tlb.c                   |   14 +-
 arch/powerpc/include/asm/kvm_book3s.h |    4 -
 arch/powerpc/include/asm/kvm_ppc.h    |    2 
 arch/powerpc/kvm/book3s.c             |    6 -
 arch/powerpc/kvm/book3s_32_mmu_host.c |    2 
 arch/powerpc/kvm/book3s_64_mmu_host.c |    2 
 arch/powerpc/kvm/e500.h               |    2 
 arch/powerpc/kvm/e500_mmu_host.c      |    8 +
 arch/powerpc/kvm/trace_pr.h           |    2 
 arch/powerpc/sysdev/axonram.c         |   11 +
 arch/um/include/asm/page.h            |    6 -
 arch/um/include/asm/pgtable-3level.h  |    4 -
 arch/um/include/asm/pgtable.h         |    2 
 arch/x86/include/asm/cacheflush.h     |    4 +
 arch/x86/include/asm/pmem.h           |    7 -
 arch/x86/kvm/iommu.c                  |   11 +
 arch/x86/kvm/mmu.c                    |   37 +++--
 arch/x86/kvm/mmu_audit.c              |    2 
 arch/x86/kvm/paging_tmpl.h            |    6 -
 arch/x86/kvm/vmx.c                    |    2 
 arch/x86/kvm/x86.c                    |    2 
 block/blk-core.c                      |   13 +-
 block/blk-mq.c                        |   19 ++-
 block/blk.h                           |   13 --
 block/ioctl.c                         |   42 ++++++
 drivers/block/brd.c                   |   10 +
 drivers/nvdimm/pmem.c                 |  247 ++++++++++++++++++++++++++++-----
 drivers/s390/block/dcssblk.c          |   13 +-
 fs/block_dev.c                        |  159 ++++++++++++++++++---
 fs/dax.c                              |  242 ++++++++++++++++++++++----------
 include/linux/blkdev.h                |   41 +++++
 include/linux/fs.h                    |   11 +
 include/linux/io.h                    |   17 --
 include/linux/kvm_host.h              |   37 +++--
 include/linux/kvm_types.h             |    2 
 include/linux/mm.h                    |   90 ++++++++++++
 include/linux/pfn.h                   |    9 +
 include/uapi/linux/fs.h               |    2 
 kernel/memremap.c                     |   98 +++++++++++++
 virt/kvm/kvm_main.c                   |   47 +++---
 45 files changed, 949 insertions(+), 325 deletions(-)

[PATCH v3 05/15] libnvdimm, pmem: fix size trim in pmem_direct_access()

[Dan Williams]

This masking prevents access to the end of the device via dax_do_io(),
and is unnecessary as arch_add_memory() would have rejected an unaligned
allocation.

Cc: <stable@vger.kernel.org>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 drivers/nvdimm/pmem.c |   17 +++--------------
 1 file changed, 3 insertions(+), 14 deletions(-)

diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index e46988fbdee5..93472953e231 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -100,26 +100,15 @@ static int pmem_rw_page(struct block_device *bdev, sector_t sector,
 }
 
 static long pmem_direct_access(struct block_device *bdev, sector_t sector,
-		      void __pmem **kaddr, unsigned long *pfn)
+		      void __pmem **kaddr, pfn_t *pfn)
 {
 	struct pmem_device *pmem = bdev->bd_disk->private_data;
 	resource_size_t offset = sector * 512 + pmem->data_offset;
-	resource_size_t size;
 
-	if (pmem->data_offset) {
-		/*
-		 * Limit the direct_access() size to what is covered by
-		 * the memmap
-		 */
-		size = (pmem->size - offset) & ~ND_PFN_MASK;
-	} else
-		size = pmem->size - offset;
-
-	/* FIXME convert DAX to comprehend that this mapping has a lifetime */
 	*kaddr = pmem->virt_addr + offset;
-	*pfn = (pmem->phys_addr + offset) >> PAGE_SHIFT;
+	*pfn = __phys_to_pfn(pmem->phys_addr + offset, pmem->pfn_flags);
 
-	return size;
+	return pmem->size - offset;
 }
 
 static const struct block_device_operations pmem_fops = {

Re: [PATCH v3 05/15] libnvdimm, pmem: fix size trim in pmem_direct_access()

[Ross Zwisler]

On Sun, Nov 01, 2015 at 11:30:10PM -0500, Dan Williams wrote:
> This masking prevents access to the end of the device via dax_do_io(),
> and is unnecessary as arch_add_memory() would have rejected an unaligned
> allocation.
> 
> Cc: <stable@vger.kernel.org>
> Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
>  drivers/nvdimm/pmem.c |   17 +++--------------
>  1 file changed, 3 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
> index e46988fbdee5..93472953e231 100644
> --- a/drivers/nvdimm/pmem.c
> +++ b/drivers/nvdimm/pmem.c
> @@ -100,26 +100,15 @@ static int pmem_rw_page(struct block_device *bdev, sector_t sector,
>  }
>  
>  static long pmem_direct_access(struct block_device *bdev, sector_t sector,
> -		      void __pmem **kaddr, unsigned long *pfn)
> +		      void __pmem **kaddr, pfn_t *pfn)

It seems kind of weird to change only this instance of direct_access() to have
the last argument as a pfn_t instead of an unsigned long?   If pfn_t is more
descriptive (I think it is) should we update the definition in struct
block_device_operations and all the other implementors of direct_access as
well?  If that's touching too much, let's do them all together later, but
let's not change one now and have them be inconsistent.

>  {
>  	struct pmem_device *pmem = bdev->bd_disk->private_data;
>  	resource_size_t offset = sector * 512 + pmem->data_offset;
> -	resource_size_t size;
>  
> -	if (pmem->data_offset) {
> -		/*
> -		 * Limit the direct_access() size to what is covered by
> -		 * the memmap
> -		 */
> -		size = (pmem->size - offset) & ~ND_PFN_MASK;
> -	} else
> -		size = pmem->size - offset;
> -
> -	/* FIXME convert DAX to comprehend that this mapping has a lifetime */
>  	*kaddr = pmem->virt_addr + offset;
> -	*pfn = (pmem->phys_addr + offset) >> PAGE_SHIFT;
> +	*pfn = __phys_to_pfn(pmem->phys_addr + offset, pmem->pfn_flags);

__phys_to_pfn() only takes a single argument (the paddr) in v4.3,
jens/for-4.4/integrity and in nvdimm/libnvdimm-for-next.  Is this second
argument of pfn_flags actually correct?

Re: [PATCH v3 05/15] libnvdimm, pmem: fix size trim in pmem_direct_access()

[Dan Williams]

On Tue, Nov 3, 2015 at 11:32 AM, Ross Zwisler
<ross.zwisler@linux.intel.com> wrote:
> On Sun, Nov 01, 2015 at 11:30:10PM -0500, Dan Williams wrote:
>> This masking prevents access to the end of the device via dax_do_io(),
>> and is unnecessary as arch_add_memory() would have rejected an unaligned
>> allocation.
>>
>> Cc: <stable@vger.kernel.org>
>> Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>> ---
>>  drivers/nvdimm/pmem.c |   17 +++--------------
>>  1 file changed, 3 insertions(+), 14 deletions(-)
>>
>> diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
>> index e46988fbdee5..93472953e231 100644
>> --- a/drivers/nvdimm/pmem.c
>> +++ b/drivers/nvdimm/pmem.c
>> @@ -100,26 +100,15 @@ static int pmem_rw_page(struct block_device *bdev, sector_t sector,
>>  }
>>
>>  static long pmem_direct_access(struct block_device *bdev, sector_t sector,
>> -                   void __pmem **kaddr, unsigned long *pfn)
>> +                   void __pmem **kaddr, pfn_t *pfn)
>
> It seems kind of weird to change only this instance of direct_access() to have
> the last argument as a pfn_t instead of an unsigned long?   If pfn_t is more
> descriptive (I think it is) should we update the definition in struct
> block_device_operations and all the other implementors of direct_access as
> well?  If that's touching too much, let's do them all together later, but
> let's not change one now and have them be inconsistent.
>

Oh, nice catch, that's just a mistake when I moved this patch earlier
in the series... and I wonder why 0day didn't complain about it?  In
any event, will fix.

>>  {
>>       struct pmem_device *pmem = bdev->bd_disk->private_data;
>>       resource_size_t offset = sector * 512 + pmem->data_offset;
>> -     resource_size_t size;
>>
>> -     if (pmem->data_offset) {
>> -             /*
>> -              * Limit the direct_access() size to what is covered by
>> -              * the memmap
>> -              */
>> -             size = (pmem->size - offset) & ~ND_PFN_MASK;
>> -     } else
>> -             size = pmem->size - offset;
>> -
>> -     /* FIXME convert DAX to comprehend that this mapping has a lifetime */
>>       *kaddr = pmem->virt_addr + offset;
>> -     *pfn = (pmem->phys_addr + offset) >> PAGE_SHIFT;
>> +     *pfn = __phys_to_pfn(pmem->phys_addr + offset, pmem->pfn_flags);
>
> __phys_to_pfn() only takes a single argument (the paddr) in v4.3,
> jens/for-4.4/integrity and in nvdimm/libnvdimm-for-next.  Is this second
> argument of pfn_flags actually correct?

Yeah, this shouldn't even compile.

Thanks Ross!