From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755945AbbJITwH (ORCPT ); Fri, 9 Oct 2015 15:52:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54519 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750987AbbJITwF (ORCPT ); Fri, 9 Oct 2015 15:52:05 -0400 Date: Fri, 9 Oct 2015 15:52:03 -0400 From: Mike Snitzer To: Mikulas Patocka Cc: Jens Axboe , kent.overstreet@gmail.com, dm-devel@redhat.com, linux-kernel@vger.kernel.org, "Alasdair G. Kergon" Subject: Re: [PATCH v2] block: flush queued bios when the process blocks Message-ID: <20151009195203.GA18790@redhat.com> References: <5384B26D.1000703@kernel.dk> <5384CE82.90601@kernel.dk> <20151005205943.GB25762@redhat.com> <20151006185016.GA31955@redhat.com> <20151006201637.GA4158@redhat.com> <20151008150859.GA11770@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151008150859.GA11770@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Oct 08 2015 at 11:08am -0400, Mike Snitzer wrote: > On Thu, Oct 08 2015 at 11:04am -0400, > Mikulas Patocka wrote: > > > > > > > On Tue, 6 Oct 2015, Mike Snitzer wrote: > > > > > To give others context for why I'm caring about this issue again, this > > > recent BZ against 4.3-rc served as a reminder that we _need_ a fix: > > > https://bugzilla.redhat.com/show_bug.cgi?id=1267650 > > > > > > FYI, I cleaned up the plug-based approach a bit further, here is the > > > incremental patch: > > > http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=wip&id=f73d001ec692125308accbb5ca26f892f949c1b6 > > > > > > And here is a new version of the overall combined patch (sharing now > > > before I transition to looking at alternatives, though my gut is the use > > > of a plug in generic_make_request really wouldn't hurt us.. famous last > > > words): > > > > > > block/bio.c | 82 +++++++++++++------------------------------------- > > > block/blk-core.c | 21 ++++++++----- > > > drivers/md/dm-bufio.c | 2 +- > > > drivers/md/raid1.c | 6 ++-- > > > drivers/md/raid10.c | 6 ++-- > > > include/linux/blkdev.h | 11 +++++-- > > > include/linux/sched.h | 4 --- > > > 7 files changed, 51 insertions(+), 81 deletions(-) > > > > ... > > > diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c > > > index 2dd3308..c2bff16 100644 > > > --- a/drivers/md/dm-bufio.c > > > +++ b/drivers/md/dm-bufio.c > > > @@ -168,7 +168,7 @@ static inline int dm_bufio_cache_index(struct dm_bufio_client *c) > > > #define DM_BUFIO_CACHE(c) (dm_bufio_caches[dm_bufio_cache_index(c)]) > > > #define DM_BUFIO_CACHE_NAME(c) (dm_bufio_cache_names[dm_bufio_cache_index(c)]) > > > > > > -#define dm_bufio_in_request() (!!current->bio_list) > > > +#define dm_bufio_in_request() (current->plug && !!current->plug->bio_list) > > > > This condition is repeated several times throughout the whole patch - so > > maybe you should make it a function in block device header file. > > Yeah, I thought of that too but forgot to come back to it. Will do, > thanks. > > FYI, I found another bug in my last patch and fixed it up. I'll get > some refactoring done (including your suggestion), actually _test_ the > code (e.g. verify all of lvm testsuite passes) and then send out v3. Turns out that this change: http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=wip&id=2639638c77768a86216be456c2764e32a2bcd841 needed to be reverted with: http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=wip&id=ad3ccd760da7c05b90775372f9b39dc2964086fe Because nested plugs caused generic_make_request()'s onstack bio_list to go out of scope (blk_finish_plug() wouldn't actually flush the list within generic_make_request because XFS already added an outermost plug). But even after fixing that I then hit issues with these changes now resulting in imperfect 'in_generic_make_request' accounting that happens lazily once the outermost plug completes blk_finish_plug. manifested as dm-bufio.c:dm_bufio_prefetch's BUG_ON(dm_bufio_in_request()); hitting. Basically using the blk-core's onstack plugging isn't workable for fixing this deadlock and we're back to having to seriously consider this (with its additional hook in the scheduler): Mike