RE: stuck iscsi/iser target with linux-4.15.0-rc1

["Steve Wise" <swise@opengridcomputing.com>]

> > [239800.115739] target_wait_for_sess_cmds: Waiting for se_cmd:
> ffff88034082c998 t_state: 6, fabric state: 12
> 
> Hmm, this means that the command was delegated to isert to send
> data+response... Which means we lose a reference put somewhere here.
> 
> I'm assuming that this happens before your changes to ib_drain_qp
> correct? If this does not happen without your changes it might indicate
> that drain_qp is missing an error (or successful?) completion which
> would prevent a final reference drop (isert_completion_put).

The issue doesn't happen on 4.15-rc4 with mlx4, but it does with cxgb4.  There have been some bad problems with cxgb4's drain logic, so I need to just root cause this, because backing up to see if it used to work is problematic and will hit other issues with drain.  The qp does get drained, and I confirmed it has no pending send or recv wrs.  The associated cq's last 2 completions are from the ib_drain_qp() logic, so I need to somehow determine what completion got lost/dropped/whatever. 

Do you have any thoughts on debugging this?  Turning on isert debug is bad because the IO load is very high.  I sort of need something that I can do after it gets stuck.  So I can change the wait_for_completion() to a wait_for_completion_timeout() and if it times out, go figure out what it is stuck on.  



---
This email has been checked for viruses by AVG.
http://www.avg.com