From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757112Ab1GMSAw (ORCPT ); Wed, 13 Jul 2011 14:00:52 -0400 Received: from 0122700014.0.fullrate.dk ([95.166.99.235]:49561 "EHLO kernel.dk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756997Ab1GMSAu (ORCPT ); Wed, 13 Jul 2011 14:00:50 -0400 Message-ID: <4E1DDD4E.6040100@kernel.dk> Date: Wed, 13 Jul 2011 20:00:46 +0200 From: Jens Axboe MIME-Version: 1.0 To: Matthew Wilcox CC: "Jiang, Dave" , "Williams, Dan J" , "Foong, Annie" , "linux-scsi@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "Nadolski, Edmund" , "Skirvin, Jeffrey D" Subject: Re: rq_affinity doesn't seem to work? References: <4E1CAEEB.8050506@kernel.dk> <20110713171011.GA11864@parisc-linux.org> In-Reply-To: <20110713171011.GA11864@parisc-linux.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2011-07-13 19:10, Matthew Wilcox wrote: > On Tue, Jul 12, 2011 at 10:30:35PM +0200, Jens Axboe wrote: >> It's probably the grouping, we need to do something about that. Does the >> below patch make it behave as you expect? > > "something", absolutely. But there is benefit from doing some aggregation > (we tried disabling it entirely with the "well-known OLTP benchmark" and > performance went down). Yep, that's why the current solution is somewhat middle of the road... > Ideally we'd do something like "if the softirq is taking up more than 10% > of a core, split the grouping". Do we have enough stats to do that kind > of monitoring? I don't think we have those stats, though it could/should be pulled from the ksoftirqX threads. We could have some metric, ala dest_cpu = get_group_completion_cpu(rq->cpu); if (ksoftirqd_of(dest_cpu) >= 90% busy) dest_cpu = rq->cpu; to send things completely local to the submitter of the IO, IFF the current CPU is close to running at full tilt. -- Jens Axboe