From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751970AbdJBQ2E (ORCPT ); Mon, 2 Oct 2017 12:28:04 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:12505 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751932AbdJBQ2C (ORCPT ); Mon, 2 Oct 2017 12:28:02 -0400 X-IronPort-AV: E=Sophos;i="5.42,470,1500912000"; d="scan'208";a="55651918" From: Bart Van Assche To: "linux-scsi@vger.kernel.org" , "hch@infradead.org" , "jejb@linux.vnet.ibm.com" , "linux-block@vger.kernel.org" , "axboe@fb.com" , "ming.lei@redhat.com" , "martin.petersen@oracle.com" CC: Bart Van Assche , "martin@lichtvoll.de" , "linux-kernel@vger.kernel.org" , "jthumshirn@suse.de" , "oleksandr@natalenko.name" , "hare@suse.com" , "cavery@redhat.com" Subject: Re: [PATCH V7 5/6] block: support PREEMPT_ONLY Thread-Topic: [PATCH V7 5/6] block: support PREEMPT_ONLY Thread-Index: AQHTObNByC6qOtWN902xf4lJZT+p4aLQw2oA Date: Mon, 2 Oct 2017 16:27:59 +0000 Message-ID: <1506961678.3368.29.camel@wdc.com> References: <20170930061214.10622-1-ming.lei@redhat.com> <20170930061214.10622-6-ming.lei@redhat.com> In-Reply-To: <20170930061214.10622-6-ming.lei@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [63.163.107.100] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;CY1PR0401MB1534;20:+USp5R+lW/JQ/NfgEqQesCnmPKZRLy4kcghdPyHrke212Cy2il4NTqtGg3kJAw/rZsJ4p1wLVCbdTGoZG5QHH/E3Dz7iWCBQDnxR+Ay/fgz6KvWjuIlQ4dof02c1N4k7dK1IFVBfVF78uIVoatXgKoO1yD7jcG4maaCeyHc2tFQ= x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-ms-office365-filtering-correlation-id: ce9e3cc7-8f4e-4bea-4d67-08d509b28bfe x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075);SRVR:CY1PR0401MB1534; x-ms-traffictypediagnostic: CY1PR0401MB1534: wdcipoutbound: EOP-TRUE x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123562025)(20161123560025)(20161123558100)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:CY1PR0401MB1534;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:CY1PR0401MB1534; x-forefront-prvs: 0448A97BF2 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(376002)(346002)(39860400002)(24454002)(199003)(189002)(377424004)(6486002)(77096006)(2900100001)(2501003)(3846002)(6116002)(86362001)(102836003)(97736004)(33646002)(8936002)(229853002)(25786009)(105586002)(106356001)(81166006)(8676002)(2950100002)(36756003)(6246003)(6436002)(6506006)(305945005)(478600001)(2906002)(72206003)(53936002)(2201001)(3660700001)(316002)(7736002)(6512007)(81156014)(189998001)(4326008)(68736007)(3280700002)(5660300001)(110136005)(66066001)(103116003)(101416001)(50986999)(54356999)(99286003)(54906003)(7416002)(76176999)(14454004);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR0401MB1534;H:CY1PR0401MB1536.namprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Oct 2017 16:27:59.8602 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB1534 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id v92GS9TN031287 On Sat, 2017-09-30 at 14:12 +0800, Ming Lei wrote: > +void blk_set_preempt_only(struct request_queue *q, bool preempt_only) > +{ > + blk_mq_freeze_queue(q); > + if (preempt_only) > + queue_flag_set_unlocked(QUEUE_FLAG_PREEMPT_ONLY, q); > + else > + queue_flag_clear_unlocked(QUEUE_FLAG_PREEMPT_ONLY, q); > + blk_mq_unfreeze_queue(q); > +} > +EXPORT_SYMBOL(blk_set_preempt_only); > + > /** > * __blk_run_queue_uncond - run a queue whether or not it has been stopped > * @q: The queue to run > @@ -771,9 +782,18 @@ int blk_queue_enter(struct request_queue *q, unsigned flags) > while (true) { > int ret; > > + /* > + * preempt_only flag has to be set after queue is frozen, > + * so it can be checked here lockless and safely > + */ > + if (blk_queue_preempt_only(q)) { > + if (!(flags & BLK_REQ_PREEMPT)) > + goto slow_path; > + } > + > if (percpu_ref_tryget_live(&q->q_usage_counter)) > return 0; Sorry but I don't think that it is possible with these changes to prevent that a non-preempt request gets allocated after a (SCSI) queue has been quiesced. If the CPU that calls blk_queue_enter() observes the set of the PREEMPT_ONLY flag after the queue has been unfrozen and after the SCSI device state has been changed into QUIESCED then blk_queue_enter() can succeed for a non-preempt request. I think this is exactly the scenario we want to avoid. This is why a synchronize_rcu() call is present in my patch before the queue is unfrozen and also why in my patch the percpu_ref_tryget_live() call occurs before the test of the PREEMPT_ONLY flag. Bart.