From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeff Moyer Subject: Re: 2.6.30-rc deadline scheduler performance regression for iozone over NFS Date: Wed, 13 May 2009 15:06:55 -0400 Message-ID: References: <20090508120119.8c93cfd7.akpm@linux-foundation.org> <20090511081415.GL4694@kernel.dk> <20090511165826.GG4694@kernel.dk> <20090512204433.7eb69075.akpm@linux-foundation.org> <20090513093229.097b47d2.akpm@linux-foundation.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Cc: Andrew Morton , Jens Axboe , linux-kernel@vger.kernel.org, "Rafael J. Wysocki" , "J. Bruce Fields" , Jim Rees , linux-nfs@vger.kernel.org To: Olga Kornievskaia Return-path: Received: from mx2.redhat.com ([66.187.237.31]:36387 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755610AbZEMTIr convert rfc822-to-8bit (ORCPT ); Wed, 13 May 2009 15:08:47 -0400 In-Reply-To: (Olga Kornievskaia's message of "Wed, 13 May 2009 14:16:42 -0400") Sender: linux-nfs-owner@vger.kernel.org List-ID: Olga Kornievskaia writes: > On Wed, May 13, 2009 at 12:32 PM, Andrew Morton > wrote: >> On Wed, 13 May 2009 12:20:57 -0400 Olga Kornievskaia wrote: >> >>> I believe what you are seeing is how well TCP autotuning performs. >>> What old NFS code was doing is disabling autotuning and instead usi= ng >>> #nfsd thread to scale TCP recv window. You are providing an example= of >>> where setting TCP buffer sizes outperforms TCP autotuning. While th= is >>> is a valid example, there is also an alternative example of where o= ld >>> NFS design hurts performance. >> >> >> >> Jeff's computer got slower. =C2=A0Can we fix that? > > We realize that decrease performance is a problem and understand that > reverting the patch might be the appropriate course of action! I wasn't suggesting that we just revert the patch. I was just looking for some guidance on diagnosing and hopefully fixing the regression. > But we are curious why this is happening. Jeff if it's not too much t= rouble > could you generate tcpdumps for both cases. We are curious what are > the max window sizes in both cases? Also could you give us your tcp a= nd > network sysctl values for the testing environment (both client and se= rver > values) that you can get with "sysctl -a | grep tcp" and also > " | grep net.core". http://people.redhat.com/jmoyer/iozone-regression.tar I'm happy to continue to help track this down. If you want to reproduc= e this in your own environment, though, you can probably do it with a ramdisk served up via nfs with the nfs client and server on the same gig-e network. > Poor performance using TCP autotuning can be demonstrated outside > of NFS but using Iperf. It can be shown that iperf will work better i= f "-w" > flag is used. When this flag is set, Iperf calls setsockopt() call wh= ich in > the kernel turns off autotuning. > > As for fixing this it would be great if we could get some help from t= he > TCP kernel folks? Then we'll need to add netdev to the CC, but probably from a message that has more background on the problem (we've even trimmed the offending commit and performance numbers from the email at this point). Cheers, Jeff