From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757190AbbIXRtn (ORCPT ); Thu, 24 Sep 2015 13:49:43 -0400 Received: from mail-yk0-f177.google.com ([209.85.160.177]:34930 "EHLO mail-yk0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754904AbbIXRtm (ORCPT ); Thu, 24 Sep 2015 13:49:42 -0400 Date: Thu, 24 Sep 2015 13:49:38 -0400 From: Tejun Heo To: Bart Van Assche Cc: Ming Lei , Jens Axboe , Christoph Hellwig , "linux-kernel@vger.kernel.org" , Akinobu Mita Subject: Re: [PATCH 3/3] blk-mq: Fix the queue freezing mechanism Message-ID: <20150924174938.GC25415@mtj.duckdns.org> References: <560323AB.80900@sandisk.com> <56032432.6080006@sandisk.com> <20150924112251.2ec061fd@tom-T450> <56042844.60603@sandisk.com> <20150924165354.GB25415@mtj.duckdns.org> <5604346D.3080000@sandisk.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5604346D.3080000@sandisk.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, Bart. On Thu, Sep 24, 2015 at 10:35:41AM -0700, Bart Van Assche wrote: > My interpretation of the percpu_ref_tryget_live() implementation in > is that the tryget operation will only fail if the > refcount is in atomic mode and additionally the __PERCPU_REF_DEAD flag has > been set. Yeah and percpu_ref_kill() does both. > >Also, what does the barriers do in your patch? > > My intention was to guarantee that on architectures that do not provide the > same ordering guarantees as x86 (e.g. PPC or ARM) that the store and load > operations on mq_freeze_depth and mq_usage_counter would not be reordered. > However, it is probably safe to leave out the barrier I proposed to > introduce in blk_mq_queue_enter() since it is acceptable that there is some > delay in communicating mq_freeze_depth updates from the CPU that modified > that counter to the CPU that reads that counter. Hmmm... please don't use barriers this way. Use it only when there's a clear requirement for interlocking writer and reader pair. There isn't one here. All it does is confusing people trying to read the code. > >The only race condition that I can see there is if unfreeze and freeze > >race each other and freeze tries to kill the ref which hasn't finished > >reinit yet. We prolly want to put mutexes around freeze/unfreeze so > >that they're serialized if something like that can happen (it isn't a > >hot path to begin with). > > My concern is that the following could happen if mq_freeze_depth is not > checked in the hot path of blk_mq_queue_enter(): > * mq_usage_counter >= 1 before blk_mq_freeze_queue() is called. > * blk_mq_freeze_queue() keeps waiting forever if new requests are queued > faster than that these requests complete. Again, that doesn't happen. Thanks. -- tejun