From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752286AbdJLPdi (ORCPT ); Thu, 12 Oct 2017 11:33:38 -0400 Received: from esa5.hgst.iphmx.com ([216.71.153.144]:18995 "EHLO esa5.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751040AbdJLPdg (ORCPT ); Thu, 12 Oct 2017 11:33:36 -0400 X-IronPort-AV: E=Sophos;i="5.43,366,1503331200"; d="scan'208";a="57639090" From: Bart Van Assche To: "osandov@osandov.com" , "ming.lei@redhat.com" CC: "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "hch@infradead.org" , "tom81094@gmail.com" , "paolo.valente@linaro.org" , "snitzer@redhat.com" , Bart Van Assche , "axboe@fb.com" , "linux-scsi@vger.kernel.org" , "oleksandr@natalenko.name" , "osandov@fb.com" , "loberman@redhat.com" , "dm-devel@redhat.com" Subject: Re: [PATCH V6 4/5] blk-mq-sched: improve dispatching from sw queue Thread-Topic: [PATCH V6 4/5] blk-mq-sched: improve dispatching from sw queue Thread-Index: AQHTQPFOhtVw1DoG50qEroTHI3G7CqLdZ+6AgAKYOwCAAFzfAA== Date: Thu, 12 Oct 2017 15:33:33 +0000 Message-ID: <1507822412.2448.12.camel@wdc.com> References: <20171009112424.30524-1-ming.lei@redhat.com> <20171009112424.30524-5-ming.lei@redhat.com> <20171010182345.GD30738@vader.DHCP.thefacebook.com> <20171012100107.GA28224@ming.t460p> In-Reply-To: <20171012100107.GA28224@ming.t460p> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [63.163.107.100] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;CY1PR0401MB1536;20:4hhxAUa6pp+eYVrT3u1cbLSr6szn2WMCG1tK+6Ak1utO3GwvGTjkGqPfLyuRL4sAOkQebQZw1pMVmO3p36IM0RFIE0HBgd1sR2husyTOnfTHHLv2rmiVShJgJJLLUaoDkIkpYFlB0PA1JwjDK8po8K552crrAKVnXOeKmlnNbOg= x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-ms-office365-filtering-correlation-id: c11cb296-9088-420f-674b-08d511869963 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075);SRVR:CY1PR0401MB1536; x-ms-traffictypediagnostic: CY1PR0401MB1536: wdcipoutbound: EOP-TRUE x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(100000703101)(100105400095)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123558100)(20161123564025)(20161123562025)(20161123560025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:CY1PR0401MB1536;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:CY1PR0401MB1536; x-forefront-prvs: 04583CED1A x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(346002)(376002)(39860400002)(377424004)(24454002)(189002)(199003)(2950100002)(36756003)(106356001)(5660300001)(3846002)(81156014)(6486002)(102836003)(81166006)(105586002)(50986999)(39060400002)(4326008)(33646002)(7416002)(6116002)(966005)(4001150100001)(77096006)(6246003)(76176999)(97736004)(2900100001)(229853002)(99286003)(14454004)(103116003)(54356999)(54906003)(93886005)(478600001)(72206003)(101416001)(8676002)(316002)(7736002)(3280700002)(305945005)(6306002)(189998001)(6512007)(3660700001)(6436002)(66066001)(25786009)(6506006)(68736007)(8936002)(53936002)(2906002)(2501003)(110136005)(230783001)(86362001);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR0401MB1536;H:CY1PR0401MB1536.namprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Oct 2017 15:33:33.7792 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB1536 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id v9CFXgVb016063 On Thu, 2017-10-12 at 18:01 +0800, Ming Lei wrote: > Even EWMA approach isn't good on SCSI-MQ too, because > some SCSI's .cmd_per_lun is very small, such as 3 on > lpfc and qla2xxx, and one full flush will trigger > BLK_STS_RESOURCE easily. > > So I suggest to use the way of q->queue_depth first, since we > don't get performance degrade report on other devices(!q->queue_depth) > with blk-mq. We can improve this way in the future if we > have better approach. Measurements have shown that even with this patch series applied sequential I/O performance is still below that of the legacy block and SCSI layers. So this patch series is not the final solution. (See also John Garry's e-mail of October 10th - https://lkml.org/lkml/2017/10/10/401). I have been wondering what could be causing that performance difference. Maybe it's because requests can reside for a while in the hctx dispatch queue and hence are unvisible for the scheduler while in the hctx dispatch queue? Should we modify blk_mq_dispatch_rq_list() such that it puts back requests that have not been accepted by .queue_rq() onto the scheduler queue(s) instead of to the hctx dispatch queue? If we would make that change, would it allow us to drop patch "blk-mq-sched: improve dispatching from sw queue"? Bart.