From mboxrd@z Thu Jan 1 00:00:00 1970 From: Junichi Nomura Subject: Re: [GIT PULL] Core block IO bits for 4.2 Date: Fri, 26 Jun 2015 06:15:12 +0000 Message-ID: <20150626061510.GA15036@xzibit.linux.bs1.fc.nec.co.jp> References: <20150625143716.GA13739@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 8BIT Return-path: In-Reply-To: <20150625143716.GA13739@kernel.dk> Content-Language: ja-JP Content-ID: <9435C87A9B9A0B4A84F5D31022C095BF@gisp.nec.co.jp> Sender: linux-kernel-owner@vger.kernel.org To: Jens Axboe , "torvalds@linuxfoundation.org" Cc: "linux-kernel@vger.kernel.org" , Christoph Hellwig , Mike Snitzer , device-mapper development List-Id: dm-devel.ids Hi Jens, On 06/25/15 23:37, Jens Axboe wrote: > block, dm: don't copy bios for request clones this change should not be pushed to mainline yet. Firstly, Christoph has a newer version of the patch that fixes silent data corruption problem: https://www.redhat.com/archives/dm-devel/2015-May/msg00229.html And the new version still depends on LLDDs to always complete requests to the end when error happens, while block API doesn't enforce such a requirement. If the assumption is ever broken, the inconsistency between request and bio (e.g. rq->__sector and rq->bio) will cause silent data corruption: https://www.redhat.com/archives/dm-devel/2015-June/msg00022.html -- Jun'ichi Nomura, NEC Corporation From mboxrd@z Thu Jan 1 00:00:00 1970 From: Linus Torvalds Subject: Re: [GIT PULL] Core block IO bits for 4.2 Date: Thu, 25 Jun 2015 23:41:06 -0700 Message-ID: References: <20150625143716.GA13739@kernel.dk> <20150626061510.GA15036@xzibit.linux.bs1.fc.nec.co.jp> Reply-To: device-mapper development Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0557951563339818264==" Return-path: In-Reply-To: <20150626061510.GA15036@xzibit.linux.bs1.fc.nec.co.jp> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com To: Junichi Nomura Cc: Jens Axboe , dm-devel@redhat.com, Christoph Hellwig , Mike Snitzer , "linux-kernel@vger.kernel.org" List-Id: dm-devel.ids --===============0557951563339818264== Content-Type: multipart/alternative; boundary=001a11c2d4624bb5220519660664 --001a11c2d4624bb5220519660664 Content-Type: text/plain; charset=UTF-8 On Jun 25, 2015 23:17, "Junichi Nomura" wrote: > > Hi Jens, > > On 06/25/15 23:37, Jens Axboe wrote: > > block, dm: don't copy bios for request clones > > this change should not be pushed to mainline yet. Hmm. I merged all of Jens' pull requests today, so it's now in my tree. What do? Linus --001a11c2d4624bb5220519660664 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Jun 25, 2015 23:17, "Junichi Nomura" <j-nomura@ce.jp.nec.com> wrote:
>
> Hi Jens,
>
> On 06/25/15 23:37, Jens Axboe wrote:
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0block, dm: don't copy bios for requ= est clones
>
> this change should not be pushed to mainline yet.

Hmm. I merged all of Jens' pull requests today, so it= 9;s now in my tree. What do?

=C2=A0=C2=A0=C2=A0=C2=A0 Linus

--001a11c2d4624bb5220519660664-- --===============0557951563339818264== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============0557951563339818264==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Junichi Nomura Subject: Re: [GIT PULL] Core block IO bits for 4.2 Date: Fri, 26 Jun 2015 08:05:35 +0000 Message-ID: <20150626080533.GA16471@xzibit.linux.bs1.fc.nec.co.jp> References: <20150625143716.GA13739@kernel.dk> <20150626061510.GA15036@xzibit.linux.bs1.fc.nec.co.jp> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 8BIT Return-path: In-Reply-To: Content-Language: ja-JP Content-ID: Sender: linux-kernel-owner@vger.kernel.org To: Linus Torvalds Cc: "dm-devel@redhat.com" , Christoph Hellwig , "linux-kernel@vger.kernel.org" , Mike Snitzer , Jens Axboe List-Id: dm-devel.ids On 06/26/15 15:41, Linus Torvalds wrote: > On Jun 25, 2015 23:17, "Junichi Nomura" > wrote: >> On 06/25/15 23:37, Jens Axboe wrote: >> > block, dm: don't copy bios for request clones >> >> this change should not be pushed to mainline yet. > > Hmm. I merged all of Jens' pull requests today, so it's now in my tree. What do? Then I think following 3 commits should be reverted. 2d76fff dm: cleanup methods that requeue requests cbc4e3c dm: do not allocate any mempools for blk-mq request-based DM 5f1b670 block, dm: don't copy bios for request clones Reverting 5f1b670 involves conflicts in drvers/md/dm.c for these 2 hunks due to diff contexts difference. It's ok to revert those changes ignoring the contexts. @@ -1089,8 +1038,6 @@ static void free_rq_clone(struct request *clone, bool must_be_mapped) WARN_ON_ONCE(must_be_mapped && !clone->q); - blk_rq_unprep_clone(clone); - if (md->type == DM_TYPE_MQ_REQUEST_BASED) /* stacked on blk-mq queue(s) */ tio->ti->type->release_clone_rq(clone); @@ -1973,11 +1889,7 @@ static int map_request(struct dm_rq_target_io *tio, struct request *rq, } if (IS_ERR(clone)) return DM_MAPIO_REQUEUE; - if (setup_clone(clone, rq, tio, GFP_ATOMIC)) { - /* -ENOMEM */ - ti->type->release_clone_rq(clone); - return DM_MAPIO_REQUEUE; - } + setup_clone(clone, rq, tio); } switch (r) { -- Jun'ichi Nomura, NEC Corporation From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mike Snitzer Subject: [git pull] device mapper fixes for 4.2-rc1 Date: Fri, 26 Jun 2015 11:20:04 -0400 Message-ID: <20150626152004.GA62003@redhat.com> References: <20150625143716.GA13739@kernel.dk> <20150626061510.GA15036@xzibit.linux.bs1.fc.nec.co.jp> <20150626080533.GA16471@xzibit.linux.bs1.fc.nec.co.jp> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20150626080533.GA16471@xzibit.linux.bs1.fc.nec.co.jp> Sender: linux-kernel-owner@vger.kernel.org To: Linus Torvalds , Junichi Nomura Cc: "dm-devel@redhat.com" , Christoph Hellwig , "linux-kernel@vger.kernel.org" , Jens Axboe List-Id: dm-devel.ids On Fri, Jun 26 2015 at 4:05P -0400, Junichi Nomura wrote: > On 06/26/15 15:41, Linus Torvalds wrote: > > On Jun 25, 2015 23:17, "Junichi Nomura" > wrote: > >> On 06/25/15 23:37, Jens Axboe wrote: > >> > block, dm: don't copy bios for request clones > >> > >> this change should not be pushed to mainline yet. > > > > Hmm. I merged all of Jens' pull requests today, so it's now in my tree. What do? > > Then I think following 3 commits should be reverted. > > 2d76fff dm: cleanup methods that requeue requests > cbc4e3c dm: do not allocate any mempools for blk-mq request-based DM > 5f1b670 block, dm: don't copy bios for request clones > > Reverting 5f1b670 involves conflicts in drvers/md/dm.c for > these 2 hunks due to diff contexts difference. > It's ok to revert those changes ignoring the contexts. That is more than I'd care to put on Linus at this point. I've prepared a DM pull request that fixes things up. Apologies for not pressing this request-based DM partial completion issue further, it was an oversight on my part. We'll have to get it fixed up properly and revisit for a future release. Linus, The following changes since commit e262f34741522e0d821642e5449c6eeb512723fc: dm stats: add support for request-based DM devices (2015-06-17 12:40:41 -0400) are available in the git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm tags/dm-4.2-fixes for you to fetch changes up to b5451e456840af027b794afc2c7c84c2a17f569b: dm cache policy smq: fix "default" version to be 1.4.0 (2015-06-26 10:14:28 -0400) ---------------------------------------------------------------- - Revert block and DM core changes that removed request-based DM's ability to handle partial request completions -- otherwise with the current SCSI LLDs these changes could lead to silent data corruption. - Fix two DM version bumps that were missing from the initial 4.2 DM pull request (enabled userspace lvm2 to know certain changes have been made). ---------------------------------------------------------------- Mike Snitzer (4): Revert "dm: do not allocate any mempools for blk-mq request-based DM" Revert "block, dm: don't copy bios for request clones" dm: bump the ioctl version to 4.32.0 dm cache policy smq: fix "default" version to be 1.4.0 block/blk-core.c | 94 +++++++++++++++-- drivers/md/dm-cache-policy-smq.c | 2 +- drivers/md/dm-table.c | 29 ++---- drivers/md/dm.c | 220 +++++++++++++++++++++++++++------------ drivers/md/dm.h | 5 +- include/linux/blk_types.h | 2 - include/linux/blkdev.h | 6 +- include/uapi/linux/dm-ioctl.h | 4 +- 8 files changed, 256 insertions(+), 106 deletions(-) From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: [GIT PULL] Core block IO bits for 4.2 Date: Fri, 26 Jun 2015 13:22:33 -0600 Message-ID: <558DA679.7000304@fb.com> References: <20150625143716.GA13739@kernel.dk> <20150626061510.GA15036@xzibit.linux.bs1.fc.nec.co.jp> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20150626061510.GA15036@xzibit.linux.bs1.fc.nec.co.jp> Sender: linux-kernel-owner@vger.kernel.org To: Junichi Nomura , "torvalds@linuxfoundation.org" Cc: "linux-kernel@vger.kernel.org" , Christoph Hellwig , Mike Snitzer , device-mapper development List-Id: dm-devel.ids On 06/26/2015 12:15 AM, Junichi Nomura wrote: > Hi Jens, > > On 06/25/15 23:37, Jens Axboe wrote: >> block, dm: don't copy bios for request clones > > this change should not be pushed to mainline yet. > > Firstly, Christoph has a newer version of the patch that fixes silent > data corruption problem: > https://www.redhat.com/archives/dm-devel/2015-May/msg00229.html > > And the new version still depends on LLDDs to always complete requests > to the end when error happens, while block API doesn't enforce such a > requirement. If the assumption is ever broken, the inconsistency between > request and bio (e.g. rq->__sector and rq->bio) will cause silent data > corruption: > https://www.redhat.com/archives/dm-devel/2015-June/msg00022.html Why was none of that sent to the block tree? In any case, Linus, pick up Mike's pull request to fix this up. -- Jens Axboe From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christoph Hellwig Subject: Re: [GIT PULL] Core block IO bits for 4.2 Date: Mon, 29 Jun 2015 19:00:50 +0200 Message-ID: <20150629170050.GA2027@lst.de> References: <20150625143716.GA13739@kernel.dk> <20150626061510.GA15036@xzibit.linux.bs1.fc.nec.co.jp> <558DA679.7000304@fb.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <558DA679.7000304@fb.com> Sender: linux-kernel-owner@vger.kernel.org To: Jens Axboe Cc: Junichi Nomura , "torvalds@linuxfoundation.org" , "linux-kernel@vger.kernel.org" , Christoph Hellwig , Mike Snitzer , device-mapper development List-Id: dm-devel.ids On Fri, Jun 26, 2015 at 01:22:33PM -0600, Jens Axboe wrote: > Why was none of that sent to the block tree? > > In any case, Linus, pick up Mike's pull request to fix this up. I didn't manage to driver the discussion forward, and hadn't really grasped it was in the block tree. For some reason I assumed it only was in Mikes dm WIP tree. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752252AbbFYOhf (ORCPT ); Thu, 25 Jun 2015 10:37:35 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:61339 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751232AbbFYOhV (ORCPT ); Thu, 25 Jun 2015 10:37:21 -0400 Date: Thu, 25 Jun 2015 08:37:16 -0600 From: Jens Axboe To: CC: Subject: [GIT PULL] Core block IO bits for 4.2 Message-ID: <20150625143716.GA13739@kernel.dk> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline X-Originating-IP: [192.168.54.13] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151,1.0.33,0.0.0000 definitions=2015-06-25_05:2015-06-25,2015-06-25,1970-01-01 signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Linus, This is the core block pull request for 4.2. Nothing really major in here, mostly a collection of smaller optimizations and cleanups, mixed with various fixes. In more detail, this pull request contains: - Addition of policy specific data to blkcg for block cgroups. From Arianna Avanzini. - Various cleanups around command types from Christoph. - Cleanup of the suspend block I/O path from Christoph. - Plugging updates from Shaohua and Jeff Moyer, for blk-mq. - Eliminating atomic inc/dec of both remaining IO count and reference count in a bio. From me. - Fixes for SG gap and chunk size support for data-less (discards) IO, so we can merge these better. From me. - Small restructuring of blk-mq shared tag support, freeing drivers from iterating hardware queues. From Keith Busch. - A few cfq-iosched tweaks, from Tahsin Erdogan and me. Makes the IOPS mode the default for non-rotational storage. Please pull! git://git.kernel.dk/linux-block.git for-4.2/core ---------------------------------------------------------------- Arianna Avanzini (1): block, cgroup: implement policy-specific per-blkcg data Christoph Hellwig (11): block: rename REQ_TYPE_SPECIAL to REQ_TYPE_DRV_PRIV block: move REQ_TYPE_ATA_TASKFILE and REQ_TYPE_ATA_PC to ide.h block: move REQ_TYPE_SENSE to the ide driver block: remove REQ_TYPE_PM_SHUTDOWN block: move PM request support to IDE nbd: stop using req->cmd block: use an atomic_t for mq_freeze_depth block: remove BIO_EOPNOTSUPP block: remove unused BIO_RW_BLOCK and BIO_EOF flags suspend: simplify block I/O handling block, dm: don't copy bios for request clones Jarod Wilson (1): block: export blkdev_reread_part() and __blkdev_reread_part() Jeff Moyer (1): blk-mq: fix plugging in blk_sq_make_request Jens Axboe (10): bio: skip atomic inc/dec of ->bi_remaining for non-chains bio: skip atomic inc/dec of ->bi_cnt for most use cases block: collapse bio bit space block: only honor SG gap prevention for merges that contain data block: don't honor chunk sizes for data-less IO block: add blk_set_queue_dying() to blkdev.h cfq-iosched: fix the setting of IOPS mode on SSDs cfq-iosched: move group scheduling functions under ifdef cfq-iosched: fix sysfs oops when attempting to read unconfigured weights cfq-iosched: fix other locations where blkcg_to_cfqgd() can return NULL Julia Lawall (1): block: fix returnvar.cocci warnings Keith Busch (1): blk-mq: Shared tag enhancements Mike Snitzer (1): block: remove management of bi_remaining when restoring original bi_end_io Ming Lei (1): block: replace trylock with mutex_lock in blkdev_reread_part() Shaohua Li (5): blk: clean up plug sched: always use blk_schedule_flush_plug in io_schedule_out blk-mq: avoid re-initialize request which is failed in direct dispatch blk-mq: do limited block plug for multiple queue case blk-mq: make plug work for mutiple disks and queues Steven Rostedt (1): blktrace: Add blktrace.c to BLOCK LAYER in MAINTAINERS file Tahsin Erdogan (1): block: Make CFQ default to IOPS mode on SSDs MAINTAINERS | 1 + block/bio-integrity.c | 4 +- block/bio.c | 77 +++++++++++-------- block/blk-cgroup.c | 92 ++++++++++++++++++++--- block/blk-cgroup.h | 40 ++++++++-- block/blk-core.c | 136 +++++++++------------------------- block/blk-exec.c | 10 --- block/blk-merge.c | 3 +- block/blk-mq-tag.c | 38 ++++++++++ block/blk-mq-tag.h | 1 + block/blk-mq.c | 160 +++++++++++++++++++++++++--------------- block/blk.h | 5 +- block/bounce.c | 3 - block/cfq-iosched.c | 125 +++++++++++++++++++++++++------ block/elevator.c | 2 + block/ioctl.c | 37 ++++++++-- drivers/block/nbd.c | 50 ++++++------- drivers/block/paride/pd.c | 4 +- drivers/block/sx8.c | 4 +- drivers/block/virtio_blk.c | 6 +- drivers/ide/ide-atapi.c | 10 +-- drivers/ide/ide-cd.c | 10 +-- drivers/ide/ide-cd_ioctl.c | 2 +- drivers/ide/ide-devsets.c | 2 +- drivers/ide/ide-eh.c | 4 +- drivers/ide/ide-floppy.c | 8 +- drivers/ide/ide-io.c | 12 +-- drivers/ide/ide-ioctls.c | 2 +- drivers/ide/ide-park.c | 4 +- drivers/ide/ide-pm.c | 56 ++++++++++---- drivers/ide/ide-tape.c | 6 +- drivers/ide/ide-taskfile.c | 2 +- drivers/md/bcache/io.c | 2 +- drivers/md/bcache/request.c | 2 +- drivers/md/dm-cache-target.c | 6 -- drivers/md/dm-raid1.c | 2 - drivers/md/dm-snap.c | 1 - drivers/md/dm-table.c | 25 ++++--- drivers/md/dm-thin.c | 9 +-- drivers/md/dm-verity.c | 2 +- drivers/md/dm.c | 171 ++++++++++--------------------------------- drivers/md/dm.h | 5 +- fs/btrfs/disk-io.c | 13 +--- fs/btrfs/extent_io.c | 2 - fs/btrfs/volumes.c | 18 ++--- fs/btrfs/volumes.h | 2 - fs/buffer.c | 13 +--- fs/ext4/page-io.c | 1 - fs/nilfs2/segbuf.c | 12 --- fs/xfs/xfs_aops.c | 1 - include/linux/bio.h | 17 ++++- include/linux/blk-mq.h | 4 + include/linux/blk_types.h | 25 ++++--- include/linux/blkdev.h | 45 ++---------- include/linux/elevator.h | 2 + include/linux/fs.h | 3 + include/linux/ide.h | 27 +++++++ include/linux/swap.h | 1 - include/uapi/linux/nbd.h | 2 - kernel/power/Makefile | 3 +- kernel/power/block_io.c | 103 -------------------------- kernel/power/power.h | 9 --- kernel/power/swap.c | 159 ++++++++++++++++++++++++++++++---------- kernel/sched/core.c | 5 +- mm/page_io.c | 2 +- 65 files changed, 853 insertions(+), 757 deletions(-) delete mode 100644 kernel/power/block_io.c -- Jens Axboe