From: Kashyap Desai <kashyap.desai@broadcom.com>
To: John Garry <john.garry@huawei.com>,
Hannes Reinecke <hare@suse.com>, Ming Lei <ming.lei@redhat.com>,
Sumit Saxena <sumit.saxena@broadcom.com>,
Bart Van Assche <bvanassche@acm.org>
Cc: chenxiang <chenxiang66@hisilicon.com>,
linux-scsi@vger.kernel.org, linux-block@vger.kernel.org,
Ewan Milne <emilne@redhat.com>, Long Li <longli@microsoft.com>,
"Martin K . Petersen" <martin.petersen@oracle.com>
Subject: RE: [bug report] Hang on sync after dd
Date: Tue, 1 Dec 2020 15:56:50 +0530 [thread overview]
Message-ID: <6cd6f97324474f88a0a748e218c8dddf@mail.gmail.com> (raw)
In-Reply-To: <2847d0e1-ccb1-7be6-2456-274e41ea981b@huawei.com>
[-- Attachment #1: Type: text/plain, Size: 2875 bytes --]
> @Kashyap, have you guys tested megaraid sas much for this?
John - I tested V4 version "scsi: core: Only re-run queue in
scsi_end_request() if device queue is busy" on MR controller.
I used different reduced device queue depth (1 to 16). I can try the exact
same test case with MR controller.
>
> Thanks,
> John
>
>
> Block debugfs info is as follows:
>
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat active cpu101/ cpu96/
> cpu99/ dispatch_busy io_poll sched_tags tags busy cpu102/ cpu97/ ctx_map
> dispatched queued sched_tags_bitmap tags_bitmap cpu100/ cpu103/ cpu98/
> dispatch flags run state type estuary:/sys/kernel/debug/block/sda/hctx8$
> cat
> cpu cpu100/ cpu101/ cpu102/ cpu103/ cpu96/ cpu97/ cpu98/ cpu99/
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat cpu cpu100/ cpu101/
> cpu102/ cpu103/ cpu96/ cpu97/ cpu98/ cpu99/
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat cpu96/ completed
> default_rq_list dispatched merged poll_rq_list read_rq_list
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat cpu96/dispatched
> 0 0
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat cpu97/dispatched
> 0 0
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat cpu98/dispatched
> 0 0
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat cpu99/dispatched
> 0 0
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat cpu100/dispatched
> 3 0
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat cpu100/completed
> 2 0
> estuary:/sys/kernel/debug/block/sda/hctx8$
> estuary:/sys/kernel/debug/block/sda/hctx8$
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat state SCHED_RESTART
When I tested V3 "scsi: core: Only re-run queue in scsi_end_request() if
device queue is busy". I noticed the similar hang and that was fixed in V4
(final patch).
Let me try on MR controller one more time. Hctx state SCHED_RESTART
indicates that someone should kicked-off h/w queue but it was missed. It may
be possible that
When you revert " scsi: core: Only re-run queue in scsi_end_request() if
device queue is busy", actual race condition windows narrows and it may be
actually existing hidden issue.
> estuary:/sys/kernel/debug/block/sda/hctx8$ ls active cpu101 cpu96 cpu99
> dispatch_busy io_poll sched_tags tags busy cpu102 cpu97 ctx_map
> dispatched queued sched_tags_bitmap tags_bitmap
> cpu100 cpu103 cpu98 dispatch flags run state type
> estuary:/sys/kernel/debug/block/sda/hctx8$ cat dispatch 000000007abb596e
> {.op=FLUSH, .cmd_flags=PREFLUSH,
> .rq_flags=FLUSH_SEQ|MQ_INFLIGHT|DONTPREP, .state=idle, .tag=21,
> .internal_tag=-1, .cmd=opcode=0x35 35 00 00 00 00 00 00 00 00 00,
> .retries=0, .result = 0x0, .flags=TAGGED|INITIALIZED|3, .timeout=60.000,
If this issue is reproducible, can you check pending commands. Is there any
pattern in pending command ?
> allocated 2208.876 s ago} estuary:/sys/kernel/debug/block/sda/hctx8$
>
>
> On cpu100, it seems completed is less than number dispatched.
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4169 bytes --]
next prev parent reply other threads:[~2020-12-01 10:28 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-30 11:22 [bug report] Hang on sync after dd John Garry
2020-12-01 10:26 ` Kashyap Desai [this message]
2020-12-01 11:48 ` John Garry
2020-12-01 12:34 ` Ming Lei
2020-12-02 1:44 ` chenxiang (M)
2020-12-02 3:22 ` Ming Lei
2020-12-02 6:22 ` chenxiang (M)
2020-12-02 7:26 ` Ming Lei
2020-12-02 9:06 ` chenxiang (M)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6cd6f97324474f88a0a748e218c8dddf@mail.gmail.com \
--to=kashyap.desai@broadcom.com \
--cc=bvanassche@acm.org \
--cc=chenxiang66@hisilicon.com \
--cc=emilne@redhat.com \
--cc=hare@suse.com \
--cc=john.garry@huawei.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=martin.petersen@oracle.com \
--cc=ming.lei@redhat.com \
--cc=sumit.saxena@broadcom.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.