[PATCH 00/11] blk_integrity lifetime fixes

[PATCH 00/11] blk_integrity lifetime fixes

[Dan Williams]

Sorry, flubbed the nvme mailing list address these patches are also in
the nvdimm patchwork and cc'd to that list if you missed them...

https://lists.01.org/pipermail/linux-nvdimm/2015-October/002430.html
https://patchwork.kernel.org/project/linux-nvdimm/list/

On Tue, Oct 13, 2015@7:27 PM, Dan Williams <dan.j.williams@intel.com> wrote:
> The recent "Block integrity registration update" caused a failing
> signature that had been triggering intermittently in the libnvdimm unit
> tests to start failing reliably every run.  These tests run through all
> several blk_integrity configurations and ends up stressing the block
> device setup/teardown path.  These are incremental fixes to "Block
> integrity registration update".
>
> Patch 1: makes the failing occurrence much lower and should be folded
> into "[PATCH 2/5] block: Consolidate static integrity profile
> properties" of the "Block integrity registration update" series.
>
> Patches 2 - 6: drop now unnecessary calls to blk_integrity_unregister()
> at driver shutdown time.
>
> Patches 7 - 8: quiesce i/o while disabling blk_integrity
>
> Patch 9: pre-cursor for patches 10, 11 extends queue lifetime guarantees
> to bio-based drivers.
>
> Patch 10: move blk_integrity to the request_queue since the usage has
> more similar lifetime to the queue than the disk
>
> Patch 11: stop-gap fix to guarantee that synchronous bio-based drivers
> wait for queued bio integrity work at shutdown.
>
> ---
>
> Dan Williams (11):
>       libnvdimm: fix blk_integrity profile allocation
>       nvme: drop blk_integrity_unregister() at shutdown
>       libnvdimm, btt: drop blk_integrity_unregister() at shutdown
>       dm: drop blk_integrity_unregister() at shutdown
>       md: drop blk_integrity_unregister() at shutdown
>       scsi: drop blk_integrity_unregister() at shutdown
>       md: suspend i/o during runtime blk_integrity_unregister
>       nvme: suspend i/o during runtime blk_integrity_unregister
>       block: generic request_queue reference counting
>       block: move blk_integrity to request_queue
>       block: blk_flush_integrity() for bio-based drivers
>
>
>  block/bio-integrity.c     |    5 +++
>  block/blk-core.c          |   74 +++++++++++++++++++++++++++++++++++++-----
>  block/blk-integrity.c     |   63 ++++++++++++++++++-----------------
>  block/blk-mq-sysfs.c      |    6 ---
>  block/blk-mq.c            |   80 ++++++++++++++-------------------------------
>  block/blk-sysfs.c         |    7 +++-
>  block/blk.h               |   16 +++++++++
>  block/genhd.c             |    2 -
>  block/partition-generic.c |    2 +
>  drivers/md/dm-table.c     |    4 +-
>  drivers/md/dm.c           |    2 -
>  drivers/md/md.c           |    6 ++-
>  drivers/md/multipath.c    |    2 +
>  drivers/md/raid1.c        |    2 +
>  drivers/md/raid10.c       |    2 +
>  drivers/nvdimm/btt.c      |    1 -
>  drivers/nvdimm/core.c     |    4 +-
>  drivers/nvme/host/pci.c   |   18 ++++++----
>  drivers/scsi/sd.c         |    1 -
>  drivers/scsi/sd_dif.c     |    2 +
>  fs/block_dev.c            |    2 +
>  include/linux/blk-mq.h    |    1 -
>  include/linux/blkdev.h    |   13 +++++--
>  include/linux/genhd.h     |   16 +++------
>  24 files changed, 188 insertions(+), 143 deletions(-)

[PATCH 08/11] nvme: suspend i/o during runtime blk_integrity_unregister

[Dan Williams]

On Wed, Oct 14, 2015@6:59 AM, Keith Busch <keith.busch@intel.com> wrote:
> On Tue, 13 Oct 2015, Dan Williams wrote:
>>
>> Synchronize pending i/o against a change in the integrity profile to
>> avoid the possibility of spurious integrity errors.
>
>
> Looks good. Do we need to freeze the queue if we are register an
> integrity profile as well?
>

Indeed we do, I'll refresh the patch.  Thanks for that catch.

[PATCH 08/11] nvme: suspend i/o during runtime blk_integrity_unregister

[Williams, Dan J]

On Wed, 2015-10-14@11:17 -0700, Dan Williams wrote:
> On Wed, Oct 14, 2015@6:59 AM, Keith Busch <keith.busch@intel.com> wrote:
> > On Tue, 13 Oct 2015, Dan Williams wrote:
> >>
> >> Synchronize pending i/o against a change in the integrity profile to
> >> avoid the possibility of spurious integrity errors.
> >
> >
> > Looks good. Do we need to freeze the queue if we are register an
> > integrity profile as well?
> >
> 
> Indeed we do, I'll refresh the patch.  Thanks for that catch.

I moved the freeze to cover all the queue manipulations in
nvme_revalidate_disk()

(git am --scissors)
8<----
Subject: nvme: suspend i/o during runtime blk_integrity_unregister

From: Dan Williams <dan.j.williams@intel.com>

Synchronize pending i/o against a change in the integrity profile to
avoid the possibility of spurious integrity errors.

Cc: Matthew Wilcox <willy at linux.intel.com>
Cc: Keith Busch <keith.busch at intel.com>
[keith: also protect dynamic integrity registration]
Signed-off-by: Dan Williams <dan.j.williams at intel.com>
---
 drivers/nvme/host/pci.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 65190e51aa47..5578de67f406 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2033,6 +2033,7 @@ static int nvme_revalidate_disk(struct gendisk *disk)
 	pi_type = ns->ms == sizeof(struct t10_pi_tuple) ?
 					id->dps & NVME_NS_DPS_PI_MASK : 0;
 
+	blk_mq_freeze_queue(disk->queue);
 	if (blk_get_integrity(disk) && (ns->pi_type != pi_type ||
 				ns->ms != old_ms ||
 				bs != queue_logical_block_size(disk->queue) ||
@@ -2052,6 +2053,7 @@ static int nvme_revalidate_disk(struct gendisk *disk)
 
 	if (dev->oncs & NVME_CTRL_ONCS_DSM)
 		nvme_config_discard(ns);
+	blk_mq_unfreeze_queue(disk->queue);
 
 	kfree(id);
 	return 0;

[PATCH 10/11] block: move blk_integrity to request_queue

[Dan Williams]

On Wed, Oct 14, 2015 at 3:40 PM, Martin K. Petersen
<martin.petersen@oracle.com> wrote:
>>>>>> "Dan" == Dan Williams <dan.j.williams at intel.com> writes:
>
> Dan> Given that a request_queue is pinned while i/o is in flight and
> Dan> that a gendisk is allowed to have a shorter lifetime, move
> Dan> blk_integrity to request_queue to satisfy requests arriving after
> Dan> the gendisk has been torn down.
>
> There was a really good reason why the integrity stuff was hanging off
> of gendisk but it's been so long that I can't remember what it was. And
> whatever it was, it is probably no longer valid.
>
> Do you have a git tree I can use to run my DIF/DIX qual on?

git://git.kernel.org/pub/scm/linux/kernel/git/djbw/nvdimm for-4.4/blk-integrity