Re: [dm-devel] SCSI Hardware Handler and slow failover with large number of LUNS

[Chandra Seetharaman <sekharan@us.ibm.com>]

Thanks for the response Mike.

On Mon, 2009-04-06 at 10:43 -0500, Mike Christie wrote:
> Chandra Seetharaman wrote:
> > Hello All,
> > 
> > During testing with the latest SCSI DH Handler on a rdac storage, Babu
> > found that the failover time with 100+ luns takes about 15 minutes,
> > which is not good.
> > 
> > We found that the problem is due to the fact that we serialize activate
> > in dm on the work queue.
> > 
> 
> I thought we talked about this during the review?

Yes, we did and the results were compared to the virgin code (w.r.t rdac
handler) and the results were good (also I used only 49 luns) :
http://marc.info/?l=dm-devel&m=120889858019762&w=2


> 
> > We can solve the problem in rdac handler in 2 ways
> >  1. batch up the activates (mode_selects) and send few of them.
> >  2. Do mode selects in async mode.
> 
> I think most of the ugliness in the original async mode was due to 
> trying to use the REQ_BLOCK* path. With the scsi_dh_activate path, it 
> should now be easier because in the send path we do not have to worry 
> about queue locks being held and context.
> 

little confused... we still are using REQ_TYPE_BLOCK_PC

> I think we could just use blk_execute_rq_nowait to send the IO. Then we 
> would have a workqueue/thread per something (maybe per dh module I 
> thought), that would be queued/notified when the IO completed. The 
> thread could then process the IO and handle the next stage if needed.
> 
> Why use the thread you might wonder? I think it fixes another issue with 
> the original async mode, and makes it easier if the scsi_dh module has 

can you elaborate the issue ?

> to send more IO. When using the thread it would not have to worry about 
> the queue_lock being held in the IO completion path and does not have to 
> worry about being run from more restrictive contexts.

You think queue_lock contention is an issue ?

I agree with the restrictive context issue though.

So, your suggestion is to move everything to async ?

> 
> 
> > 
> > Just wondering if anybody had seen the same problem in other storages
> > (EMC, HP and Alua). 
> 
> They should all have the same problem.
> 
> 
> > 
> > Please share your experiences, so we can come up with a solution that
> > works for all hardware handlers.
> > 
> > regards,
> > 
> > chandra
> > 
> > --
> > dm-devel mailing list
> > dm-devel@redhat.com
> > https://www.redhat.com/mailman/listinfo/dm-devel
>