From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752513Ab0CKBpr (ORCPT ); Wed, 10 Mar 2010 20:45:47 -0500 Received: from mga02.intel.com ([134.134.136.20]:11465 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750960Ab0CKBpq (ORCPT ); Wed, 10 Mar 2010 20:45:46 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.49,617,1262592000"; d="scan'208";a="603266377" Date: Thu, 11 Mar 2010 09:45:42 +0800 From: Wu Fengguang To: Christian Ehrhardt Cc: Jens Axboe , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , Martin Schwidefsky , Heiko Carstens , Hisashi Hifumi , KOSAKI Motohiro , Ronald , Bart Van Assche , Vladislav Bolkhovitin , Randy Dunlap , Nick Piggin Subject: Re: [RFC PATCH] Fix Readahead stalling by plugged device queues Message-ID: <20100311014542.GA8134@localhost> References: <4B979104.6010907@linux.vnet.ibm.com> <20100310130932.GB18509@localhost> <4B97AD52.7080201@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B97AD52.7080201@linux.vnet.ibm.com> User-Agent: Mutt/1.5.18 (2008-05-17) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Mar 10, 2010 at 10:31:46PM +0800, Christian Ehrhardt wrote: > > > Wu Fengguang wrote: > [...] > > Christian, did you notice this commit for 2.6.33? > > > > commit 65a80b4c61f5b5f6eb0f5669c8fb120893bfb388 > [...] > > I didn't see that particular one, due to the fact that whatever the > result is it needs to work .32 > > Anyway I'll test it tomorrow and if that already accepted one fixes my > issue as well I'll recommend distros older than 2.6.33 picking that one > up in their on top patches. OK, thanks! > > > > It should at least improve performance between .32 and .33, because > > once two readahead requests are merged into one single IO request, > > the PageUptodate() will be true at next readahead, and hence > > blk_run_backing_dev() get called to break out of the suboptimal > > situation. > > As you saw from my blktrace thats already the case without that patch. > Once the second readahead comes in and merged it gets unplugged in > 2.6.32 too - but still that is bad behavior as it denies my things like > 68% throughput improvement :-). I mean, when readahead windows A and B are submitted in one IO -- let's call it AB -- commit 65a80b4c61 will explicitly unplug on doing readahead C. While in your trace, the unplug appears on AB. The 68% improvement is very impressive. Wondering if commit 65a80b4c61 (the _conditional_ unplug) can achieve the same level of improvement :) > > > > Your patch does reduce the possible readahead submit latency to 0. > > yeah and I think/hope that is fine, because as I stated: > - low utilized disk -> not an issue > - high utilized disk -> unplug is an noop > > At least personally I consider a case where merging of a readahead > window with anything except its own sibling very rare - and therefore > fair to unplug after and RA is submitted. They are reasonable assumptions. However I'm not sure if this unconditional unplug will defeat CFQ's anticipatory logic -- if there are any. You know commit 65a80b4c61 is more about a *defensive* protection against the rare case that two readahead windows get merged. > > Is your workload a simple dd on a single disk? If so, it sounds like > > something illogical hidden in the block layer. > > It might still be illogical hidden as e.g. 2.6.27 unplugged after the > first readahead as well :-) > But no my load is iozone running with different numbers of processes > with one disk per process. > That neatly resembles e.g. nightly backup jobs which tend to take longer > and longer in all time increasing customer scenarios. Such an > improvement might banish the backups back to the night were they belong :-) Exactly one process per disk? Are they doing sequential reads or more complicated access patterns? Thanks, Fengguang