Re: [RFC PATCH 1/8] rqbased-dm: allow blk_get_request() to be called from interrupt context

[Jens Axboe <jens.axboe@oracle.com>]

On Wed, Dec 20 2006, Kiyoshi Ueda wrote:
> Hi Jens,
> 
> Thank you for the comment.
> 
> On Wed, 20 Dec 2006 14:48:49 +0100, Jens Axboe <jens.axboe@oracle.com> wrote:
> > >  static struct request *get_request(request_queue_t *q, int rw, struct bio *bio,
> > > -				   gfp_t gfp_mask)
> > > +				   gfp_t gfp_mask, unsigned long *flags)
> > >  {
> > >  	struct request *rq = NULL;
> > >  	struct request_list *rl = &q->rq;
> > > @@ -2119,7 +2120,10 @@ static struct request *get_request(reque
> > >  	if (priv)
> > >  		rl->elvpriv++;
> > >  
> > > -	spin_unlock_irq(q->queue_lock);
> > > +	if (flags)
> > > +		spin_unlock_irqrestore(q->queue_lock, *flags);
> > > +	else
> > > +		spin_unlock_irq(q->queue_lock);
> > 
> > Big NACK on this - it's not only really ugly, it's also buggy to pass
> > interrupt flags as function arguments. As you also mention in the 0/1
> > mail, this also breaks CFQ.
> > 
> > Why do you need in-interrupt request allocation?
>  
> Because I'd like to use blk_get_request() in q->request_fn()
> which can be called from interrupt context like below:
>   scsi_io_completion -> scsi_end_request -> scsi_next_command
>   -> scsi_run_queue -> blk_run_queue -> q->request_fn
> 
> Generally, device-mapper (dm) clones an original I/O and dispatches
> the clones to underlying destination devices.
> In the request-based dm patch, the clone creation and the dispatch
> are done in q->request_fn().  To create the clone, blk_get_request()
> is used to get a request from underlying destination device's queue.
> By doing that in q->request_fn(), dm can deal with struct request
> after bios are merged by __make_request().
> 
> Do you think creating another function like blk_get_request_nowait()
> is acceptable?
> Or request should not be allocated in q->request_fn() anyway?

You should not be allocating requests from that path, for a number of
reasons. The design isn't very nice either.

The easy way out would be to punt to a workqueue to handle the requests.

An alternative way would be to set aside some requests that you can get
at without allocation (maintain a little freelist of manually allocated
requests), and retrieve a free one from there when inside request_fn. If
you run out, just bail out of request_fn and make sure to reinvoke it
when some of your previously issued requests complete and are added back
to that freelist.

-- 
Jens Axboe