From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F21AEC433EF for ; Wed, 17 Nov 2021 09:00:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CF7E161BF5 for ; Wed, 17 Nov 2021 09:00:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231167AbhKQJDi (ORCPT ); Wed, 17 Nov 2021 04:03:38 -0500 Received: from szxga01-in.huawei.com ([45.249.212.187]:14950 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234857AbhKQJD2 (ORCPT ); Wed, 17 Nov 2021 04:03:28 -0500 Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4HvGzb5rp3zZd6F; Wed, 17 Nov 2021 16:58:03 +0800 (CST) Received: from kwepemm600019.china.huawei.com (7.193.23.64) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Wed, 17 Nov 2021 17:00:24 +0800 Received: from [10.174.177.210] (10.174.177.210) by kwepemm600019.china.huawei.com (7.193.23.64) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.15; Wed, 17 Nov 2021 17:00:23 +0800 Subject: Re: [QUESTION] blk_mq_freeze_queue in elevator_init_mq To: Ming Lei CC: , , , , , , , , , References: From: yangerkun Message-ID: Date: Wed, 17 Nov 2021 17:00:22 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.177.210] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemm600019.china.huawei.com (7.193.23.64) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 2021/11/17 16:06, Ming Lei wrote: > On Wed, Nov 17, 2021 at 11:37:13AM +0800, yangerkun wrote: >> Nowdays we meet the boot regression while enable lots of mtdblock > > What is your boot regression? Any dmesg log? The result is that when boot with 5.10 kernel compare with 4.4, 5.10 will consume about 1.6s more... > >> compare with 4.4. The main reason was that the blk_mq_freeze_queue in >> elevator_init_mq will wait a RCU gap which want to make sure no IO will >> happen while blk_mq_init_sched. > > There isn't RCU grace period implied in the blk_mq_freeze_queue() called > from elevator_init_mq(), because the .q_usage_counter works at atomic mode > at that time. > >> >> Other module like loop meets this problem too and has been fix with > > Again, what is the problem? commit 2112f5c1330a671fa852051d85cb9eadc05d7eb7 Author: Bart Van Assche Date: Thu Aug 5 10:42:00 2021 -0700 loop: Select I/O scheduler 'none' from inside add_disk() ... e.g. via a udev rule. This approach has an advantage compared to changing the I/O scheduler from userspace from 'mq-deadline' into 'none', namely that synchronize_rcu() does not get called. Actually, we has meets this problem too before(https://www.spinics.net/lists/linux-block/msg70660.html). > >> follow patches: >> >> 2112f5c1330a loop: Select I/O scheduler 'none' from inside add_disk() >> 90b7198001f2 blk-mq: Introduce the BLK_MQ_F_NO_SCHED_BY_DEFAULT flag >> >> They change the default IO scheduler for loop to 'none'. So no need to >> call blk_mq_freeze_queue and blk_mq_init_sched. But it seems not >> appropriate for mtdblocks. Mtdblocks can use 'mq-deadline' to help >> optimize the random write with the help of mtdblock's cache. Once change >> to 'none', we may meet the regression for random write. >> >> commit 737eb78e82d52d35df166d29af32bf61992de71d >> Author: Damien Le Moal >> Date: Thu Sep 5 18:51:33 2019 +0900 >> >> block: Delay default elevator initialization >> >> ... >> >> Additionally, to make sure that the elevator initialization is never >> done while requests are in-flight (there should be none when the device >> driver calls device_add_disk()), freeze and quiesce the device request >> queue before calling blk_mq_init_sched() in elevator_init_mq(). >> ... >> >> This commit add blk_mq_freeze_queue in elevator_init_mq which try to >> make sure no in-flight request while we go through blk_mq_init_sched. >> But does there any drivers can leave IO alive while we go through >> elevator_init_mq？ And if no, maybe we can just remove this logical to >> fix the regression... > > SCSI should have passthrough requests at that moment. > > > > Thanks, > Ming > > . > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DB53C433F5 for ; Wed, 17 Nov 2021 09:04:05 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9CBA26187F for ; Wed, 17 Nov 2021 09:04:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9CBA26187F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:CC:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=e1WqW/RYQnGH1Qu02hzvcB4wEYkcLCmxhsrpflA0498=; b=BHjgkkrTcHrgUcLmLrlk4kOEgk CNexU8E227BSyamBaDew5dIpF8Pp6IHLwKC0zy5moD8UOV/IlEjz9DWMicJ4D6zZGLvk7DRTfKqce Bt7aDt8dZyotbKQ7nt/8tZv6+niwXb9J1zKYD0k6voJUoiKMOUnj1p8l/nvz6eXgdgEuF5GR5PG4I KmMia4GLBkXlJKKclkS5NDgK+vZQaszqz/9z+mefxKHorc9MLjJBZFcScW+hvje1UuAODEyQ5B/we IluYIWBv1aI+J14IP2tVIYMklfLV2NUflzLfC2BKygi8H3GzuVMjCaALqBty7PDI2I+aSsrCj4Hcn d6xUWybw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mnGqg-0044My-OF; Wed, 17 Nov 2021 09:03:22 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mnGnu-0042aY-OR for linux-mtd@lists.infradead.org; Wed, 17 Nov 2021 09:00:33 +0000 Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4HvGzb5rp3zZd6F; Wed, 17 Nov 2021 16:58:03 +0800 (CST) Received: from kwepemm600019.china.huawei.com (7.193.23.64) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Wed, 17 Nov 2021 17:00:24 +0800 Received: from [10.174.177.210] (10.174.177.210) by kwepemm600019.china.huawei.com (7.193.23.64) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.15; Wed, 17 Nov 2021 17:00:23 +0800 Subject: Re: [QUESTION] blk_mq_freeze_queue in elevator_init_mq To: Ming Lei CC: , , , , , , , , , References: From: yangerkun Message-ID: Date: Wed, 17 Nov 2021 17:00:22 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Originating-IP: [10.174.177.210] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemm600019.china.huawei.com (7.193.23.64) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211117_010031_168066_920FC5F9 X-CRM114-Status: GOOD ( 21.28 ) X-BeenThere: linux-mtd@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Linux MTD discussion mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-mtd" Errors-To: linux-mtd-bounces+linux-mtd=archiver.kernel.org@lists.infradead.org CgpPbiAyMDIxLzExLzE3IDE2OjA2LCBNaW5nIExlaSB3cm90ZToKPiBPbiBXZWQsIE5vdiAxNywg MjAyMSBhdCAxMTozNzoxM0FNICswODAwLCB5YW5nZXJrdW4gd3JvdGU6Cj4+IE5vd2RheXMgd2Ug bWVldCB0aGUgYm9vdCByZWdyZXNzaW9uIHdoaWxlIGVuYWJsZSBsb3RzIG9mIG10ZGJsb2NrCj4g Cj4gV2hhdCBpcyB5b3VyIGJvb3QgcmVncmVzc2lvbj8gQW55IGRtZXNnIGxvZz8KClRoZSByZXN1 bHQgaXMgdGhhdCB3aGVuIGJvb3Qgd2l0aCA1LjEwIGtlcm5lbCBjb21wYXJlIHdpdGggNC40LCA1 LjEwCndpbGwgY29uc3VtZSBhYm91dCAxLjZzIG1vcmUuLi4KPiAKPj4gY29tcGFyZSB3aXRoIDQu NC4gVGhlIG1haW4gcmVhc29uIHdhcyB0aGF0IHRoZSBibGtfbXFfZnJlZXplX3F1ZXVlIGluCj4+ IGVsZXZhdG9yX2luaXRfbXEgd2lsbCB3YWl0IGEgUkNVIGdhcCB3aGljaCB3YW50IHRvIG1ha2Ug c3VyZSBubyBJTyB3aWxsCj4+IGhhcHBlbiB3aGlsZSBibGtfbXFfaW5pdF9zY2hlZC4KPiAKPiBU aGVyZSBpc24ndCBSQ1UgZ3JhY2UgcGVyaW9kIGltcGxpZWQgaW4gdGhlIGJsa19tcV9mcmVlemVf cXVldWUoKSBjYWxsZWQKPiBmcm9tIGVsZXZhdG9yX2luaXRfbXEoKSwgYmVjYXVzZSB0aGUgLnFf dXNhZ2VfY291bnRlciB3b3JrcyBhdCBhdG9taWMgbW9kZQo+IGF0IHRoYXQgdGltZS4KPiAKPj4K Pj4gT3RoZXIgbW9kdWxlIGxpa2UgbG9vcCBtZWV0cyB0aGlzIHByb2JsZW0gdG9vIGFuZCBoYXMg YmVlbiBmaXggd2l0aAo+IAo+IEFnYWluLCB3aGF0IGlzIHRoZSBwcm9ibGVtPwoKY29tbWl0IDIx MTJmNWMxMzMwYTY3MWZhODUyMDUxZDg1Y2I5ZWFkYzA1ZDdlYjcKQXV0aG9yOiBCYXJ0IFZhbiBB c3NjaGUgPGJ2YW5hc3NjaGVAYWNtLm9yZz4KRGF0ZTogICBUaHUgQXVnIDUgMTA6NDI6MDAgMjAy MSAtMDcwMAoKICAgICBsb29wOiBTZWxlY3QgSS9PIHNjaGVkdWxlciAnbm9uZScgZnJvbSBpbnNp ZGUgYWRkX2Rpc2soKQoKICAgICAuLi4KICAgICBlLmcuIHZpYSBhIHVkZXYgcnVsZS4gVGhpcyBh cHByb2FjaCBoYXMgYW4gYWR2YW50YWdlIGNvbXBhcmVkIHRvIApjaGFuZ2luZwogICAgIHRoZSBJ L08gc2NoZWR1bGVyIGZyb20gdXNlcnNwYWNlIGZyb20gJ21xLWRlYWRsaW5lJyBpbnRvICdub25l JywgbmFtZWx5CiAgICAgdGhhdCBzeW5jaHJvbml6ZV9yY3UoKSBkb2VzIG5vdCBnZXQgY2FsbGVk LgoKQWN0dWFsbHksIHdlIGhhcyBtZWV0cyB0aGlzIHByb2JsZW0gdG9vIApiZWZvcmUoaHR0cHM6 Ly93d3cuc3Bpbmljcy5uZXQvbGlzdHMvbGludXgtYmxvY2svbXNnNzA2NjAuaHRtbCkuCgoKPiAK Pj4gZm9sbG93IHBhdGNoZXM6Cj4+Cj4+ICAgMjExMmY1YzEzMzBhIGxvb3A6IFNlbGVjdCBJL08g c2NoZWR1bGVyICdub25lJyBmcm9tIGluc2lkZSBhZGRfZGlzaygpCj4+ICAgOTBiNzE5ODAwMWYy IGJsay1tcTogSW50cm9kdWNlIHRoZSBCTEtfTVFfRl9OT19TQ0hFRF9CWV9ERUZBVUxUIGZsYWcK Pj4KPj4gVGhleSBjaGFuZ2UgdGhlIGRlZmF1bHQgSU8gc2NoZWR1bGVyIGZvciBsb29wIHRvICdu b25lJy4gU28gbm8gbmVlZCB0bwo+PiBjYWxsIGJsa19tcV9mcmVlemVfcXVldWUgYW5kIGJsa19t cV9pbml0X3NjaGVkLiBCdXQgaXQgc2VlbXMgbm90Cj4+IGFwcHJvcHJpYXRlIGZvciBtdGRibG9j a3MuIE10ZGJsb2NrcyBjYW4gdXNlICdtcS1kZWFkbGluZScgdG8gaGVscAo+PiBvcHRpbWl6ZSB0 aGUgcmFuZG9tIHdyaXRlIHdpdGggdGhlIGhlbHAgb2YgbXRkYmxvY2sncyBjYWNoZS4gT25jZSBj aGFuZ2UKPj4gdG8gJ25vbmUnLCB3ZSBtYXkgbWVldCB0aGUgcmVncmVzc2lvbiBmb3IgcmFuZG9t IHdyaXRlLgo+Pgo+PiBjb21taXQgNzM3ZWI3OGU4MmQ1MmQzNWRmMTY2ZDI5YWYzMmJmNjE5OTJk ZTcxZAo+PiBBdXRob3I6IERhbWllbiBMZSBNb2FsIDxkYW1pZW4ubGVtb2FsQHdkYy5jb20+Cj4+ IERhdGU6ICAgVGh1IFNlcCA1IDE4OjUxOjMzIDIwMTkgKzA5MDAKPj4KPj4gICAgICBibG9jazog RGVsYXkgZGVmYXVsdCBlbGV2YXRvciBpbml0aWFsaXphdGlvbgo+Pgo+PiAgICAgIC4uLgo+Pgo+ PiAgICAgIEFkZGl0aW9uYWxseSwgdG8gbWFrZSBzdXJlIHRoYXQgdGhlIGVsZXZhdG9yIGluaXRp YWxpemF0aW9uIGlzIG5ldmVyCj4+ICAgICAgZG9uZSB3aGlsZSByZXF1ZXN0cyBhcmUgaW4tZmxp Z2h0ICh0aGVyZSBzaG91bGQgYmUgbm9uZSB3aGVuIHRoZSBkZXZpY2UKPj4gICAgICBkcml2ZXIg Y2FsbHMgZGV2aWNlX2FkZF9kaXNrKCkpLCBmcmVlemUgYW5kIHF1aWVzY2UgdGhlIGRldmljZSBy ZXF1ZXN0Cj4+ICAgICAgcXVldWUgYmVmb3JlIGNhbGxpbmcgYmxrX21xX2luaXRfc2NoZWQoKSBp biBlbGV2YXRvcl9pbml0X21xKCkuCj4+ICAgICAgLi4uCj4+Cj4+IFRoaXMgY29tbWl0IGFkZCBi bGtfbXFfZnJlZXplX3F1ZXVlIGluIGVsZXZhdG9yX2luaXRfbXEgd2hpY2ggdHJ5IHRvCj4+IG1h a2Ugc3VyZSBubyBpbi1mbGlnaHQgcmVxdWVzdCB3aGlsZSB3ZSBnbyB0aHJvdWdoIGJsa19tcV9p bml0X3NjaGVkLgo+PiBCdXQgZG9lcyB0aGVyZSBhbnkgZHJpdmVycyBjYW4gbGVhdmUgSU8gYWxp dmUgd2hpbGUgd2UgZ28gdGhyb3VnaAo+PiBlbGV2YXRvcl9pbml0X21x77yfIEFuZCBpZiBubywg bWF5YmUgd2UgY2FuIGp1c3QgcmVtb3ZlIHRoaXMgbG9naWNhbCB0bwo+PiBmaXggdGhlIHJlZ3Jl c3Npb24uLi4KPiAKPiBTQ1NJIHNob3VsZCBoYXZlIHBhc3N0aHJvdWdoIHJlcXVlc3RzIGF0IHRo YXQgbW9tZW50Lgo+IAo+IAo+IAo+IFRoYW5rcywKPiBNaW5nCj4gCj4gLgo+IAoKX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkxpbnV4IE1URCBk aXNjdXNzaW9uIG1haWxpbmcgbGlzdApodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFu L2xpc3RpbmZvL2xpbnV4LW10ZC8K