From: Bart Van Assche <Bart.VanAssche@wdc.com>
To: "axboe@kernel.dk" <axboe@kernel.dk>,
"ming.lei@redhat.com" <ming.lei@redhat.com>
Cc: "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
"hch@infradead.org" <hch@infradead.org>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"osandov@fb.com" <osandov@fb.com>,
"john.garry@huawei.com" <john.garry@huawei.com>,
"loberman@redhat.com" <loberman@redhat.com>
Subject: Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()
Date: Tue, 7 Nov 2017 17:34:38 +0000 [thread overview]
Message-ID: <1510076077.2656.24.camel@wdc.com> (raw)
In-Reply-To: <74925156-b425-de0e-de39-0a166ef5c5f3@kernel.dk>
T24gVHVlLCAyMDE3LTExLTA3IGF0IDA5OjI5IC0wNzAwLCBKZW5zIEF4Ym9lIHdyb3RlOg0KPiBP
biAxMS8wNy8yMDE3IDA5OjIwIEFNLCBCYXJ0IFZhbiBBc3NjaGUgd3JvdGU6DQo+ID4gT24gVHVl
LCAyMDE3LTExLTA3IGF0IDEwOjExICswODAwLCBNaW5nIExlaSB3cm90ZToNCj4gPiA+IElmIHlv
dSBjYW4gcmVwcm9kdWNlLCBwbGVhc2UgcHJvdmlkZSBtZSBhdCBsZWFzdCB0aGUgZm9sbG93aW5n
IGxvZw0KPiA+ID4gZmlyc3Q6DQo+ID4gPiANCj4gPiA+IAlmaW5kIC9zeXMva2VybmVsL2RlYnVn
L2Jsb2NrIC1uYW1lIHRhZ3MgfCB4YXJncyBjYXQgfCBncmVwIGJ1c3kNCj4gPiA+IA0KPiA+ID4g
SWYgYW55IHBlbmRpbmcgcmVxdWVzdHMgYXJuJ3QgY29tcGxldGVkLCBwbGVhc2UgcHJvdmlkZSB0
aGUgcmVsYXRlZA0KPiA+ID4gaW5mbyBpbiBkYmdmcyBhYm91dCB3aGVyZSBpcyB0aGUgcmVxdWVz
dC4NCj4gPiANCj4gPiBFdmVyeSB0aW1lIEkgcmFuIHRoZSBhYm92ZSBvciBhIHNpbWlsYXIgY29t
bWFuZCBpdHMgb3V0cHV0IHdhcyBlbXB0eS4gSQ0KPiA+IGFzc3VtZSB0aGF0J3MgYmVjYXVzZSB0
aGUgaGFuZyB1c3VhbGx5IG9jY3VycyBpbiBhIHBoYXNlIHdoZXJlIHRoZXNlIGRlYnVnZnMNCj4g
PiBhdHRyaWJ1dGVzIGVpdGhlciBoYXZlIG5vdCB5ZXQgYmVlbiBjcmVhdGVkIG9yIGhhdmUgYWxy
ZWFkeSBkaXNhcHBlYXJlZC4NCj4gDQo+IEJhcnQsIGRvIHlvdSBzdGlsbCBzZWUgYSBoYW5nIHdp
dGggdGhlIHBhdGNoIHRoYXQgZml4ZXMgdGhlIHRhZyBsZWFrIHdoZW4NCj4gd2UgZmFpbCB0byBn
ZXQgYSBkaXNwYXRjaCBidWRnZXQ/DQo+IA0KPiBodHRwczovL21hcmMuaW5mby8/bD1saW51eC1i
bG9jayZtPTE1MTAwNDg4MTQxMTQ4MCZ3PTINCj4gDQo+IEkndmUgcnVuIGEgbG90IG9mIHN0YWJp
bGl0eSB0ZXN0aW5nIGhlcmUsIGFuZCBJIGhhdmVuJ3QgcnVuIGludG8gYW55DQo+IGlzc3Vlcy4g
VGhpcyBpcyB3aXRoIHNoYXJlZCB0YWdzIGFzIHdlbGwuIFNvIGlmIHlvdSBzdGlsbCBzZWUgdGhl
IGZhaWx1cmUNCj4gY2FzZSB3aXRoIHRoZSBjdXJyZW50IHRyZWUgQU5EIHRoZSBhYm92ZSBwYXRj
aCwgdGhlbiBJJ2xsIHRyeSBhbmQgZ2V0DQo+IGEgdGVzdCBjYXNlIHNldHVwIHRoYXQgaGl0cyBp
dCB0b28gc28gd2UgY2FuIGdldCB0byB0aGUgYm90dG9tIG9mIHRoaXMuDQoNCkl0IHRvb2sgYSBs
aXR0bGUgbG9uZ2VyIHRoYW4gZXhwZWN0ZWQgYnV0IEkganVzdCByYW4gaW50byB0aGUgZm9sbG93
aW5nDQpsb2NrdXAgd2l0aCB5b3VyIGZvci1uZXh0IGJyYW5jaCBvZiB0aGlzIG1vcm5pbmcgKGNv
bW1pdCBlOGZhNDRiYjhhZjkpIGFuZA0KTWluZydzIHBhdGNoICJibGstbXE6IHB1dCBkcml2ZXIg
dGFnIGlmIGRpc3BhdGNoIGJ1ZGdldCBjYW4ndCBiZSBnb3QiDQphcHBsaWVkIG9uIHRvcCBvZiBp
dDoNCg0KWyAyNTc1LjMyNDY3OF0gc3lzcnE6IFN5c1JxIDogU2hvdyBCbG9ja2VkIFN0YXRlDQpb
IDI1NzUuMzMyMzM2XSAgIHRhc2sgICAgICAgICAgICAgICAgICAgICAgICBQQyBzdGFjayAgIHBp
ZCBmYXRoZXINClsgMjU3NS4zNDUyMzldIHN5c3RlbWQtdWRldmQgICBEICAgIDAgNDc1NzcgICAg
NTE4IDB4MDAwMDAxMDYNClsgMjU3NS4zNTM4MjFdIENhbGwgVHJhY2U6DQpbIDI1NzUuMzU4ODA1
XSAgX19zY2hlZHVsZSsweDI4Yi8weDg5MA0KWyAyNTc1LjM2NDkwNl0gIHNjaGVkdWxlKzB4MzYv
MHg4MA0KWyAyNTc1LjM3MDQzNl0gIGlvX3NjaGVkdWxlKzB4MTYvMHg0MA0KWyAyNTc1LjM3NjI4
N10gIF9fbG9ja19wYWdlKzB4ZmMvMHgxNDANClsgMjU3NS4zODIwNjFdICA/IHBhZ2VfY2FjaGVf
dHJlZV9pbnNlcnQrMHhjMC8weGMwDQpbIDI1NzUuMzg4OTQzXSAgdHJ1bmNhdGVfaW5vZGVfcGFn
ZXNfcmFuZ2UrMHg1ZTgvMHg4MzANClsgMjU3NS4zOTYwODNdICB0cnVuY2F0ZV9pbm9kZV9wYWdl
cysweDE1LzB4MjANClsgMjU3NS40MDIzOThdICBraWxsX2JkZXYrMHgyZi8weDQwDQpbIDI1NzUu
NDA3NTM4XSAgX19ibGtkZXZfcHV0KzB4NzQvMHgxZjANClsgMjU3NS40MTMwMTBdICA/IGttZW1f
Y2FjaGVfZnJlZSsweDE5Ny8weDFjMA0KWyAyNTc1LjQxODk4Nl0gIGJsa2Rldl9wdXQrMHg0Yy8w
eGQwDQpbIDI1NzUuNDI0MDQwXSAgYmxrZGV2X2Nsb3NlKzB4MzQvMHg3MA0KWyAyNTc1LjQyOTIx
Nl0gIF9fZnB1dCsweGU3LzB4MjIwDQpbIDI1NzUuNDMzODYzXSAgX19fX2ZwdXQrMHhlLzB4MTAN
ClsgMjU3NS40Mzg0OTBdICB0YXNrX3dvcmtfcnVuKzB4NzYvMHg5MA0KWyAyNTc1LjQ0MzYxOV0g
IGRvX2V4aXQrMHgyZTAvMHhhZjANClsgMjU3NS40NDgzMTFdICBkb19ncm91cF9leGl0KzB4NDMv
MHhiMA0KWyAyNTc1LjQ1MzM4Nl0gIGdldF9zaWduYWwrMHgyOTkvMHg1ZTANClsgMjU3NS40NTgz
MDNdICBkb19zaWduYWwrMHgzNy8weDc0MA0KWyAyNTc1LjQ2Mjk3Nl0gID8gYmxrZGV2X3JlYWRf
aXRlcisweDM1LzB4NDANClsgMjU3NS40Njg0MjVdICA/IG5ld19zeW5jX3JlYWQrMHhkZS8weDEz
MA0KWyAyNTc1LjQ3MzYyMF0gID8gdmZzX3JlYWQrMHgxMTUvMHgxMzANClsgMjU3NS40NzgzODhd
ICBleGl0X3RvX3VzZXJtb2RlX2xvb3ArMHg4MC8weGQwDQpbIDI1NzUuNDg0MDAyXSAgZG9fc3lz
Y2FsbF82NCsweGIzLzB4YzANClsgMjU3NS40ODg4MTNdICBlbnRyeV9TWVNDQUxMNjRfc2xvd19w
YXRoKzB4MjUvMHgyNQ0KWyAyNTc1LjQ5NDc1OV0gUklQOiAwMDMzOjB4N2VmZDgyOWNiZDExDQpb
IDI1NzUuNDk5NTA2XSBSU1A6IDAwMmI6MDAwMDdmZmZmOTg0Zjk3OCBFRkxBR1M6IDAwMDAwMjQ2
IE9SSUdfUkFYOiAwMDAwMDAwMDAwMDAwMDAwDQpbIDI1NzUuNTA4NzQxXSBSQVg6IDAwMDAwMDAw
MDAwMjIwMDAgUkJYOiAwMDAwNTVmMTlmOTAyY2EwIFJDWDogMDAwMDdlZmQ4MjljYmQxMQ0KWyAy
NTc1LjUxNzQ1NV0gUkRYOiAwMDAwMDAwMDAwMDQwMDAwIFJTSTogMDAwMDU1ZjE5ZjkwMmNjOCBS
REk6IDAwMDAwMDAwMDAwMDAwMDcNClsgMjU3NS41MjYxNjNdIFJCUDogMDAwMDU1ZjE5ZjdmYjlk
MCBSMDg6IDAwMDAwMDAwMDAwMDAwMDAgUjA5OiAwMDAwNTVmMTlmOTAyY2EwDQpbIDI1NzUuNTM0
ODYwXSBSMTA6IDAwMDA1NWYxOWY5MDJjYjggUjExOiAwMDAwMDAwMDAwMDAwMjQ2IFIxMjogMDAw
MDAwMDAwMDAwMDAwMA0KWyAyNTc1LjU0NDI1MF0gUjEzOiAwMDAwMDAwMDAwMDQwMDAwIFIxNDog
MDAwMDU1ZjE5ZjdmYmEyMCBSMTU6IDAwMDAwMDAwMDAwNDAwMDANCg0KQmFydC4=
WARNING: multiple messages have this Message-ID (diff)
From: Bart Van Assche <Bart.VanAssche@wdc.com>
To: "axboe@kernel.dk" <axboe@kernel.dk>,
"ming.lei@redhat.com" <ming.lei@redhat.com>
Cc: "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
"hch@infradead.org" <hch@infradead.org>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"osandov@fb.com" <osandov@fb.com>,
"john.garry@huawei.com" <john.garry@huawei.com>,
"loberman@redhat.com" <loberman@redhat.com>
Subject: Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()
Date: Tue, 7 Nov 2017 17:34:38 +0000 [thread overview]
Message-ID: <1510076077.2656.24.camel@wdc.com> (raw)
In-Reply-To: <74925156-b425-de0e-de39-0a166ef5c5f3@kernel.dk>
On Tue, 2017-11-07 at 09:29 -0700, Jens Axboe wrote:
> On 11/07/2017 09:20 AM, Bart Van Assche wrote:
> > On Tue, 2017-11-07 at 10:11 +0800, Ming Lei wrote:
> > > If you can reproduce, please provide me at least the following log
> > > first:
> > >
> > > find /sys/kernel/debug/block -name tags | xargs cat | grep busy
> > >
> > > If any pending requests arn't completed, please provide the related
> > > info in dbgfs about where is the request.
> >
> > Every time I ran the above or a similar command its output was empty. I
> > assume that's because the hang usually occurs in a phase where these debugfs
> > attributes either have not yet been created or have already disappeared.
>
> Bart, do you still see a hang with the patch that fixes the tag leak when
> we fail to get a dispatch budget?
>
> https://marc.info/?l=linux-block&m=151004881411480&w=2
>
> I've run a lot of stability testing here, and I haven't run into any
> issues. This is with shared tags as well. So if you still see the failure
> case with the current tree AND the above patch, then I'll try and get
> a test case setup that hits it too so we can get to the bottom of this.
It took a little longer than expected but I just ran into the following
lockup with your for-next branch of this morning (commit e8fa44bb8af9) and
Ming's patch "blk-mq: put driver tag if dispatch budget can't be got"
applied on top of it:
[ 2575.324678] sysrq: SysRq : Show Blocked State
[ 2575.332336] task PC stack pid father
[ 2575.345239] systemd-udevd D 0 47577 518 0x00000106
[ 2575.353821] Call Trace:
[ 2575.358805] __schedule+0x28b/0x890
[ 2575.364906] schedule+0x36/0x80
[ 2575.370436] io_schedule+0x16/0x40
[ 2575.376287] __lock_page+0xfc/0x140
[ 2575.382061] ? page_cache_tree_insert+0xc0/0xc0
[ 2575.388943] truncate_inode_pages_range+0x5e8/0x830
[ 2575.396083] truncate_inode_pages+0x15/0x20
[ 2575.402398] kill_bdev+0x2f/0x40
[ 2575.407538] __blkdev_put+0x74/0x1f0
[ 2575.413010] ? kmem_cache_free+0x197/0x1c0
[ 2575.418986] blkdev_put+0x4c/0xd0
[ 2575.424040] blkdev_close+0x34/0x70
[ 2575.429216] __fput+0xe7/0x220
[ 2575.433863] ____fput+0xe/0x10
[ 2575.438490] task_work_run+0x76/0x90
[ 2575.443619] do_exit+0x2e0/0xaf0
[ 2575.448311] do_group_exit+0x43/0xb0
[ 2575.453386] get_signal+0x299/0x5e0
[ 2575.458303] do_signal+0x37/0x740
[ 2575.462976] ? blkdev_read_iter+0x35/0x40
[ 2575.468425] ? new_sync_read+0xde/0x130
[ 2575.473620] ? vfs_read+0x115/0x130
[ 2575.478388] exit_to_usermode_loop+0x80/0xd0
[ 2575.484002] do_syscall_64+0xb3/0xc0
[ 2575.488813] entry_SYSCALL64_slow_path+0x25/0x25
[ 2575.494759] RIP: 0033:0x7efd829cbd11
[ 2575.499506] RSP: 002b:00007ffff984f978 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 2575.508741] RAX: 0000000000022000 RBX: 000055f19f902ca0 RCX: 00007efd829cbd11
[ 2575.517455] RDX: 0000000000040000 RSI: 000055f19f902cc8 RDI: 0000000000000007
[ 2575.526163] RBP: 000055f19f7fb9d0 R08: 0000000000000000 R09: 000055f19f902ca0
[ 2575.534860] R10: 000055f19f902cb8 R11: 0000000000000246 R12: 0000000000000000
[ 2575.544250] R13: 0000000000040000 R14: 000055f19f7fba20 R15: 0000000000040000
Bart.
next prev parent reply other threads:[~2017-11-07 17:34 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-04 1:55 [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget() Ming Lei
2017-11-04 14:19 ` Jens Axboe
2017-11-06 19:45 ` Bart Van Assche
2017-11-06 19:45 ` Bart Van Assche
2017-11-07 2:11 ` Ming Lei
2017-11-07 16:20 ` Bart Van Assche
2017-11-07 16:20 ` Bart Van Assche
2017-11-07 16:29 ` Jens Axboe
2017-11-07 17:10 ` Jens Axboe
2017-11-07 17:36 ` Jens Axboe
2017-11-07 22:06 ` Jens Axboe
2017-11-07 22:34 ` Bart Van Assche
2017-11-07 22:34 ` Bart Van Assche
2017-11-07 22:39 ` Jens Axboe
2017-11-08 0:50 ` Ming Lei
2017-11-08 1:03 ` Ming Lei
2017-11-08 3:01 ` Jens Axboe
2017-11-08 3:12 ` Ming Lei
2017-11-08 3:17 ` Jens Axboe
2017-11-08 3:17 ` Jens Axboe
2017-11-08 6:20 ` Ming Lei
2017-11-08 15:59 ` Ming Lei
2017-11-08 18:19 ` Jens Axboe
2017-11-07 17:34 ` Bart Van Assche [this message]
2017-11-07 17:34 ` Bart Van Assche
2017-11-08 0:53 ` Ming Lei
2017-11-08 2:06 ` Ming Lei
2017-11-08 0:39 ` Ming Lei
2017-11-08 2:55 ` Jens Axboe
2017-11-08 2:58 ` Ming Lei
2017-11-08 3:06 ` Jens Axboe
2017-11-08 16:41 ` Bart Van Assche
2017-11-08 16:41 ` Bart Van Assche
2017-11-08 17:57 ` Jens Axboe
2017-11-08 18:22 ` Laurence Oberman
2017-11-08 18:28 ` Jens Axboe
2017-11-09 4:02 ` Ming Lei
2017-11-09 2:05 ` Ming Lei
2017-11-07 10:15 ` Ming Lei
2017-11-07 16:17 ` Bart Van Assche
2017-11-07 16:17 ` Bart Van Assche
2017-11-08 3:12 ` Jens Axboe
2017-11-06 18:04 ` Bart Van Assche
2017-11-06 18:04 ` Bart Van Assche
2017-11-07 2:19 ` Ming Lei
2017-11-07 3:53 ` Martin K. Petersen
2017-11-07 3:53 ` Martin K. Petersen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1510076077.2656.24.camel@wdc.com \
--to=bart.vanassche@wdc.com \
--cc=axboe@kernel.dk \
--cc=hch@infradead.org \
--cc=john.garry@huawei.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=loberman@redhat.com \
--cc=ming.lei@redhat.com \
--cc=osandov@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.