From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tejun Heo Subject: Re: [PATCH 07/11] blkcg: make request_queue bypassing on allocation Date: Fri, 13 Apr 2012 13:37:26 -0700 Message-ID: <20120413203726.GE12233@google.com> References: <1334347895-6268-1-git-send-email-tj@kernel.org> <1334347895-6268-8-git-send-email-tj@kernel.org> <20120413203205.GI26383@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=JxOFLs4i6CU2Ij8NblAUXwLuvVPj8U0TH90BSU26tZs=; b=I9r9QvYXbY8YVfhzx21iDiOgyIshiWTqzcUvjFDPLPRVk/9SeWiEZK4HHo/4eujvNs 2bNd2pFFw8gzmj/Nh23inBpGnpFhinV5VixSfngLeDnZ0o7bOaQ9uS2gCSHYlMhuF9VX nG6jKDPdK/UipZak8wFSDPKVgKn0uiZtBjDSzbVypho2i3B3NJmol99ov4gsuI9Ts63q tVnRAPZOn7hMoc5UG5H/9EhBwnoE7S/hRWD7q6Gs5qhENMahS+o4wxF//LoIRL5Jypka dDZBgnILtJMILzju/TD5nYSw8013bKH43Y37FO8IBUYKlxpx7jm88azRYQkw8mHkjT8Q uupg== Content-Disposition: inline In-Reply-To: <20120413203205.GI26383-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: containers-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: containers-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Vivek Goyal Cc: axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org, ctalbott-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, rni-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org On Fri, Apr 13, 2012 at 04:32:05PM -0400, Vivek Goyal wrote: > On Fri, Apr 13, 2012 at 01:11:31PM -0700, Tejun Heo wrote: > > With the previous change to guarantee bypass visiblity for RCU read > > lock regions, entering bypass mode involves non-trivial overhead and > > future changes are scheduled to make use of bypass mode during init > > path. Combined it may end up adding noticeable delay during boot. > > > > This patch makes request_queue start its life in bypass mode, which is > > ended on queue init completion at the end of > > blk_init_allocated_queue(), and updates blk_queue_bypass_start() such > > that draining and RCU synchronization are performed only when the > > queue actually enters bypass mode. > > > > This avoids unnecessarily switching in and out of bypass mode during > > init avoiding the overhead and any nasty surprises which may step from > > leaving bypass mode on half-initialized queues. > > Tejun, I am not sure that this will fix the problem completely. I think > we will still face the overhead of synchronize_rcu() in > blkcg_deactivate_policy() as it will be called from cfq_exit_queue() for > initialized queues. > > In the past I had I had used synchronize_rcu() in cfq_exit_queue() and > noticed the overhead. Looks like driver was creating fully initialized > queues and tearing these apart soon. blk_cleanup_queue() doesn't use blk_queue_bypass_start() to enter bypass mode. Thanks. -- tejun From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756763Ab2DMUhc (ORCPT ); Fri, 13 Apr 2012 16:37:32 -0400 Received: from mail-pz0-f52.google.com ([209.85.210.52]:35676 "EHLO mail-pz0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753013Ab2DMUha (ORCPT ); Fri, 13 Apr 2012 16:37:30 -0400 Date: Fri, 13 Apr 2012 13:37:26 -0700 From: Tejun Heo To: Vivek Goyal Cc: axboe@kernel.dk, ctalbott@google.com, rni@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, containers@lists.linux-foundation.org Subject: Re: [PATCH 07/11] blkcg: make request_queue bypassing on allocation Message-ID: <20120413203726.GE12233@google.com> References: <1334347895-6268-1-git-send-email-tj@kernel.org> <1334347895-6268-8-git-send-email-tj@kernel.org> <20120413203205.GI26383@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120413203205.GI26383@redhat.com> User-Agent: Mutt/1.5.20 (2009-06-14) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Apr 13, 2012 at 04:32:05PM -0400, Vivek Goyal wrote: > On Fri, Apr 13, 2012 at 01:11:31PM -0700, Tejun Heo wrote: > > With the previous change to guarantee bypass visiblity for RCU read > > lock regions, entering bypass mode involves non-trivial overhead and > > future changes are scheduled to make use of bypass mode during init > > path. Combined it may end up adding noticeable delay during boot. > > > > This patch makes request_queue start its life in bypass mode, which is > > ended on queue init completion at the end of > > blk_init_allocated_queue(), and updates blk_queue_bypass_start() such > > that draining and RCU synchronization are performed only when the > > queue actually enters bypass mode. > > > > This avoids unnecessarily switching in and out of bypass mode during > > init avoiding the overhead and any nasty surprises which may step from > > leaving bypass mode on half-initialized queues. > > Tejun, I am not sure that this will fix the problem completely. I think > we will still face the overhead of synchronize_rcu() in > blkcg_deactivate_policy() as it will be called from cfq_exit_queue() for > initialized queues. > > In the past I had I had used synchronize_rcu() in cfq_exit_queue() and > noticed the overhead. Looks like driver was creating fully initialized > queues and tearing these apart soon. blk_cleanup_queue() doesn't use blk_queue_bypass_start() to enter bypass mode. Thanks. -- tejun