From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755082AbeARRVF (ORCPT ); Thu, 18 Jan 2018 12:21:05 -0500 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:41423 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753228AbeARRVB (ORCPT ); Thu, 18 Jan 2018 12:21:01 -0500 X-IronPort-AV: E=Sophos;i="5.46,378,1511798400"; d="scan'208";a="69871375" From: Bart Van Assche To: "snitzer@redhat.com" CC: "dm-devel@redhat.com" , "hch@infradead.org" , "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "osandov@fb.com" , "ming.lei@redhat.com" , "axboe@kernel.dk" Subject: Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle Thread-Topic: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle Thread-Index: AQHTkAXlp7DN5JBZKUeyLfttG5XpOaN52OmAgAADroCAAATDAA== Date: Thu, 18 Jan 2018 17:20:57 +0000 Message-ID: <1516296056.2676.23.camel@wdc.com> References: <20180118024124.8079-1-ming.lei@redhat.com> <20180118170353.GB19734@redhat.com> In-Reply-To: <20180118170353.GB19734@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [199.255.44.172] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;CY1PR0401MB0971;7:CCZMvBXUcjsd/JxOFqzgFQdGmtY5rTokurdPDbfP0KQFZoAUCZMQnPscTOFIpO7q5qNkmCxVHHYxFbeFyKUUXWs9zSIcSgMTA228VtkYxF8ynzM5HsAuSqX1ebccBrOLtfoIRSUebmNTh/OpEKbZe1v/n1+gqkcC0bp4l6w8TLvLFKcGjn04wR7ltw5iQpc+BFhPQU7tLi2lSzYEPxPYY57bAkm5ex5Q7IYamtuCN1h+LrRFt2rOuhWyuGTUIahq;20:RZSRbiXAHOJTzcq//QFh2SGLXCceD84bfel4FaJBtJsfyM/hXZTEPR9o+lftZvF1O/oSDbqTWnsuh4t7SKzcehAlcFQk7T/5PJP88bLMA7u6dDKO7vC0X/qVj8XgAMI9KIrgcPFXM4wCCsv6BgPWq6SB01TAdwwnE8FcyzC2m4M= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: b9af751f-e11b-4dab-51a9-08d55e97d67c x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(3008032)(2017052603307)(7153060)(7193020);SRVR:CY1PR0401MB0971; x-ms-traffictypediagnostic: CY1PR0401MB0971: wdcipoutbound: EOP-TRUE x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(84791874153150); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(2401047)(5005006)(8121501046)(93006095)(93001095)(3231023)(2400064)(944501161)(10201501046)(3002001)(6055026)(6041268)(20161123564045)(201703131423095)(201703061421075)(20161123560045)(20161123558120)(20161123562045)(6042181)(6072148)(201708071742011);SRVR:CY1PR0401MB0971;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:CY1PR0401MB0971; x-forefront-prvs: 05568D1FF7 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(377424004)(199004)(189003)(51444003)(2950100002)(6916009)(7736002)(6246003)(2906002)(3280700002)(5660300001)(68736007)(305945005)(59450400001)(102836004)(97736004)(4326008)(54906003)(66066001)(77096007)(53936002)(86362001)(6486002)(6436002)(72206003)(14454004)(5640700003)(2351001)(26005)(105586002)(106356001)(2900100001)(508600001)(25786009)(8936002)(6306002)(76176011)(229853002)(99286004)(966005)(6506007)(3660700001)(36756003)(1730700003)(81156014)(3846002)(6512007)(103116003)(81166006)(2501003)(8676002)(6116002);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR0401MB0971;H:CY1PR0401MB1536.namprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; x-microsoft-antispam-message-info: JDgsruNXbZ68z72wDL17PDX7ryrSzFTy6t9P8oJI484AOfVhIXx9d5dQ2zKpMdqOhB9HQt2chqZoA4U3V2hjDg== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <2B12C166E6858A4BB4E6A3473F5F72B8@namprd04.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: b9af751f-e11b-4dab-51a9-08d55e97d67c X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jan 2018 17:20:57.3556 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB0971 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id w0IHLB7d027666 On Thu, 2018-01-18 at 12:03 -0500, Mike Snitzer wrote: > On Thu, Jan 18 2018 at 11:50am -0500, > Bart Van Assche wrote: > > My comments about the above are as follows: > > - It can take up to q->rq_timeout jiffies after a .queue_rq() > > implementation returned BLK_STS_RESOURCE before blk_mq_timeout_work() > > gets called. However, it can happen that only a few milliseconds after > > .queue_rq() returned BLK_STS_RESOURCE that the condition that caused > > it to return BLK_STS_RESOURCE gets cleared. So the above approach can > > result in long delays during which it will seem like the queue got > > stuck. Additionally, I think that the block driver should decide how > > long it takes before a queue is rerun and not the block layer core. > > So configure q->rq_timeout to be shorter? Which is configurable though > blk_mq_tag_set's 'timeout' member. It apparently defaults to 30 * HZ. > > That is the problem with timeouts, there is generally no one size fits > all. Sorry but I think that would be wrong. The delay after which a queue is rerun should not be coupled to the request timeout. These two should be independent. > > - The lockup that I reported only occurs with the dm driver but not any > > other block driver. So why to modify the block layer core since this > > can be fixed by modifying the dm driver? > > Hard to know it is only DM's blk-mq that is impacted. That is the only > blk-mq driver that you're testing like this (that is also able to handle > faults, etc). That's not correct. I'm also testing the SCSI core, which is one of the most complicated block drivers. > > - A much simpler fix and a fix that is known to work exists, namely > > inserting a blk_mq_delay_run_hw_queue() call in the dm driver. > > Because your "much simpler" fix actively hurts performance, as is > detailed in this header: > https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.16&id=ec3eaf9a673106f66606896aed6ddd20180b02ec We are close to the start of the merge window so I think it's better to fall back to an old approach that is known to work than to keep a new approach that is known not to work. Additionally, the performance issue you referred to only affects IOPS and bandwidth more than 1% with the lpfc driver and that is because the queue depth it supports is much lower than for other SCSI HBAs, namely 3 instead of 64. Thanks, Bart.