Re: elevator private data for REQ_FLUSH

[Mike Snitzer <snitzer@redhat.com>]

On Fri, Mar 25 2011 at  3:50pm -0400,
Jens Axboe <axboe@kernel.dk> wrote:

> On 2011-03-25 19:54, Mike Snitzer wrote:
> > On Fri, Mar 25 2011 at 11:50am -0400,
> > Jens Axboe <axboe@kernel.dk> wrote:
> > 
> >> On 2011-03-25 16:40, Mike Snitzer wrote:
> >>> On Fri, Mar 25 2011 at 11:22am -0400,
> >>> Markus Trippelsdorf <markus@trippelsdorf.de> wrote:
> >>>
> >>>> On 2011.03.25 at 17:15 +0200, Sergey Senozhatsky wrote:
> >>>>> Hello,
> >>>>>
> >>>>> Commit
> >>>>>     9d5a4e946ce5352f19400b6370f4cd8e72806278
> >>>>>     block: skip elevator data initialization for flush requests
> >>>>>     
> >>>>>     Skip elevator initialization for flush requests by passing priv=0 to
> >>>>>     blk_alloc_request() in get_request().  As such elv_set_request() is
> >>>>>     never called for flush requests.
> >>>>>
> >>>>> introduced priv flag, to skip elevator_private data init for FLUSH requests.
> >>>>> This, I guess, lead to NULL pointer deref on my machine in cfq_insert_request,
> >>>>> which requires elevator_private to be set:
> >>>>>
> >>>>>   1 [   78.982169] Call Trace:                                                                                                                                                                                                     
> >>>>>   2 [   78.982178]  [<ffffffff8122d1fe>] cfq_insert_request+0x4e/0x47d
> >>>>>   3 [   78.982184]  [<ffffffff8123e139>] ? do_raw_spin_lock+0x6b/0x122
> >>>>
> >>>>> Should we in that case use ELEVATOR_INSERT_FLUSH for REQ_FLUSH | REQ_FUA requests
> >>>>> (like below)?
> >>>>>
> >>>>> ---
> >>>>>
> >>>>>  block/elevator.c |    2 ++
> >>>>>  1 files changed, 2 insertions(+), 0 deletions(-)
> >>>>>
> >>>>> diff --git a/block/elevator.c b/block/elevator.c
> >>>>> index c387d31..b17e577 100644
> >>>>> --- a/block/elevator.c
> >>>>> +++ b/block/elevator.c
> >>>>> @@ -734,6 +734,8 @@ void __elv_add_request(struct request_queue *q, struct request *rq, int where)
> >>>>>  			q->end_sector = rq_end_sector(rq);
> >>>>>  			q->boundary_rq = rq;
> >>>>>  		}
> >>>>> +	} else if (rq->cmd_flags & (REQ_FLUSH | REQ_FUA)) {
> >>>>> +		where = ELEVATOR_INSERT_FLUSH;
> >>>>>  	} else if (!(rq->cmd_flags & REQ_ELVPRIV) &&
> >>>>>  		    where == ELEVATOR_INSERT_SORT)
> >>>>>  		where = ELEVATOR_INSERT_BACK;
> >>>>
> >>>> Thanks. That solves all (corruption-) problems that I reported earlier in an other
> >>>> thread. 
> >>>
> >>> So the flush-merge changes introduced ELEVATOR_INSERT_FLUSH (via commit
> >>> ae1b1539).  And the flush bio will now get ELEVATOR_INSERT_FLUSH in
> >>> __make_request().
> >>>
> >>> So it is interesting that the flush is getting inserted in the elevator
> >>> at all.  AFAIK that shouldn't be (and historically hasn't been) the
> >>> case.
> >>>
> >>> Combination of onstack plug changes?
> >>
> >> It is, it forces a sort insert. I'll fix this up, I'm relieved we have a
> >> good handle on this issue now.
> > 
> > Should we also add a safety net to avoid the potential for future silent
> > corruption, etc?  E.g.:
> 
> Yes, I was thinking about something like that. I consider the patch
> merged an immediate stop gap, we need to improve this situation. It's
> not exactly pretty to have this sort of condition in both
> __make_request() and flush_plug_list(). Clearly it should be handled
> further down.

OK, and btw my patch was too restrictive.  blk_kick_flush()
elv_insert()s a flush request with ELEVATOR_INSERT_REQUEUE.

Should blk_kick_flush() process the flush request without calling
elv_insert() -- like is done with open coded list_add() in
blk_insert_flush()?

Or should blk_insert_flush() use elv_insert() with
ELEVATOR_INSERT_REQUEUE too? 

Mike