Re: blk_throtl_exit taking q->queue_lock is problematic

[Mike Snitzer <snitzer@redhat.com>]

On Thu, Feb 17 2011 at 10:33pm -0500,
NeilBrown <neilb@suse.de> wrote:

> On Thu, 17 Feb 2011 22:19:52 -0500 Mike Snitzer <snitzer@redhat.com> wrote:
> 
> > On Thu, Feb 17, 2011 at 9:40 PM, NeilBrown <neilb@suse.de> wrote:
> > > On Thu, 17 Feb 2011 11:59:06 -0500 Vivek Goyal <vgoyal@redhat.com> wrote:
> > >> So if we do this change for performance reasons, it still makes sense
> > >> but doing this change because md provided a q->queue_lock and took away that
> > >> lock without notifying block layer hence we do this change, is still not
> > >> the right reason, IMHO.
> > >
> > > Well...I like that patch, as it makes my life easier....
> > >
> > > But I agree that md is doing something wrong.  Now that ->queue_lock is
> > > always initialised, it is wrong to leave it in a state where it not defined.
> > >
> > > So maybe I'll apply this (after testing it a bit.  The only reason for taking
> > > the lock queue_lock in a couple of places is to silence some warnings.
> > >
> > > Thanks,
> > > NeilBrown
> > >
> > >
> > > diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
> > > index a23ffa3..909282d 100644
> > > --- a/drivers/md/raid1.c
> > > +++ b/drivers/md/raid1.c
> > > @@ -959,7 +961,9 @@ static int make_request(mddev_t *mddev, struct bio * bio)
> > >                atomic_inc(&r1_bio->remaining);
> > >                spin_lock_irqsave(&conf->device_lock, flags);
> > >                bio_list_add(&conf->pending_bio_list, mbio);
> > > +               spin_lock(mddev->queue->queue_lock);
> > >                blk_plug_device(mddev->queue);
> > > +               spin_unlock(mddev->queue->queue_lock);
> > >                spin_unlock_irqrestore(&conf->device_lock, flags);
> > >        }
> > >        r1_bio_write_done(r1_bio, bio->bi_vcnt, behind_pages, behind_pages != NULL);
> > 
> > Noticed an inconsistency, raid10.c's additional locking also protects
> > the bio_list_add() whereas raid1.c's doesn't.  Seems the additional
> > protection in raid10 isn't needed?
> 
> Correct - not needed at all.
> I put it there because it felt a little cleaner keeping the two 'lock's
> together like the two 'unlock's.  Probably confusing though...
> 
> My other though is to stop using the block-layer plugging altogether like I
> have in RAID5 (Which I needed to do to make it work with DM).  Then I
> wouldn't need to touch queue_lock at all - very tempting.

FYI, Jens has a considerable plugging overhaul staged in his tree for 2.6.39:
http://git.kernel.dk/?p=linux-2.6-block.git;a=shortlog;h=refs/heads/for-2.6.39/stack-plug

So if you do ween MD off of the block-layer's plugging Jens will need to
adapt his patches that touch MD.. not a big deal.

Mike