From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o4S7gMYa203579 for ; Fri, 28 May 2010 02:42:23 -0500 Received: from mgw-mx06.nokia.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id AB678130BDBA for ; Fri, 28 May 2010 00:46:39 -0700 (PDT) Received: from mgw-mx06.nokia.com (smtp.nokia.com [192.100.122.233]) by cuda.sgi.com with ESMTP id cPl3IybG6zDk0J7r for ; Fri, 28 May 2010 00:46:39 -0700 (PDT) Subject: Re: [PATCH 0/5] Per superblock shrinkers V2 From: Artem Bityutskiy In-Reply-To: <20100527133223.efa4740a.akpm@linux-foundation.org> References: <1274777588-21494-1-git-send-email-david@fromorbit.com> <20100527133223.efa4740a.akpm@linux-foundation.org> Date: Fri, 28 May 2010 10:42:06 +0300 Message-ID: <1275032526.15516.83.camel@localhost> Mime-Version: 1.0 Reply-To: dedekind1@gmail.com List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: xfs-bounces@oss.sgi.com Errors-To: xfs-bounces@oss.sgi.com To: Andrew Morton Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com T24gVGh1LCAyMDEwLTA1LTI3IGF0IDEzOjMyIC0wNzAwLCBBbmRyZXcgTW9ydG9uIHdyb3RlOgo+ IE9uIFR1ZSwgMjUgTWF5IDIwMTAgMTg6NTM6MDMgKzEwMDAKPiBEYXZlIENoaW5uZXIgPGRhdmlk QGZyb21vcmJpdC5jb20+IHdyb3RlOgo+IAo+ID4gVGhpcyBzZXJpZXMgcmV3b3JrcyB0aGUgZmls ZXN5c3RlbSBzaHJpbmtlcnMuIFdlIGN1cnJlbnRseSBoYXZlIGEKPiA+IHNldCBvZiBpc3N1ZXMg d2l0aCB0aGUgY3VycmVudCBmaWxlc3lzdGVtIHNocmlua2VyczoKPiA+IAo+ID4gICAgICAgICAx LiBUaGVyZSBpcyBhbiBkZXBlbmRlbmN5IGJldHdlZW4gZGVudHJ5IGFuZCBpbm9kZSBjYWNoZQo+ ID4gICAgICAgICAgICBzaHJpbmtpbmcgdGhhdCBpcyBvbmx5IGltcGxpY2l0bHkgZGVmaW5lZCBi eSB0aGUgb3JkZXIgb2YKPiA+ICAgICAgICAgICAgc2hyaW5rZXIgcmVnaXN0cmF0aW9uLgo+ID4g ICAgICAgICAyLiBUaGUgc2hyaW5rZXJzIG5lZWQgdG8gd2FsayB0aGUgc3VwZXJibG9jayBsaXN0 IGFuZCBwaW4KPiA+ICAgICAgICAgICAgdGhlIHN1cGVyYmxvY2sgdG8gYXZvaWQgdW5tb3VudCBy YWNlcyB3aXRoIHRoZSBzYiBnb2luZwo+ID4gICAgICAgICAgICBhd2F5Lgo+ID4gICAgICAgICAz LiBUaGUgZGVudHJ5IGNhY2hlIHVzZXMgcGVyLXN1cGVyYmxvY2sgTFJVcyBhbmQgcHJvcG9ydGlv bnMKPiA+ICAgICAgICAgICAgcmVjbGFpbSBiZXR3ZWVuIGFsbCB0aGUgc3VwZXJibG9ja3Mgd2hp Y2ggbWVhbnMgd2UgYXJlCj4gPiAgICAgICAgICAgIGRvaW5nIGJyZWFkdGggYmFzZWQgcmVjbGFp bS4gVGhpcyBtZWFucyB3ZSB0b3VjaCBldmVyeQo+ID4gICAgICAgICAgICBzdXBlcmJsb2NrIGZv ciBldmVyeSBzaHJpbmtlciBjYWxsLCBhbmQgbWF5IG9ubHkgcmVjbGFpbQo+ID4gICAgICAgICAg ICBhIHNpbmdsZSBkZW50cnkgYXQgYSB0aW1lIGZyb20gYSBnaXZlbiBzdXBlcmJsb2NrLgo+ID4g ICAgICAgICA0LiBUaGUgaW5vZGUgY2FjaGUgaGFzIGEgZ2xvYmFsIExSVSwgc28gaXQgaGFzIGRp ZmZlcmVudAo+ID4gICAgICAgICAgICByZWNsYWltIHBhdHRlcm5zIHRvIHRoZSBkZW50cnkgY2Fj aGUsIGRlc3BpdGUgdGhlIGZhY3QKPiA+ICAgICAgICAgICAgdGhhdCB0aGUgZGVudHJ5IGNhY2hl IGlzIGdlbmVyYWxseSB0aGUgb25seSB0aGluZyB0aGF0Cj4gPiAgICAgICAgICAgIHBpbnMgaW5v ZGVzIGluIG1lbW9yeS4KPiA+ICAgICAgICAgNS4gRmlsZXN5c3RlbXMgbmVlZCB0byByZWdpc3Rl ciB0aGVpciBvd24gc2hyaW5rZXJzIGZvcgo+ID4gICAgICAgICAgICBjYWNoZXMgYW5kIGNhbid0 IGNvLW9yZGluYXRlIHRoZW0gd2l0aCB0aGUgZGVudHJ5IGFuZAo+ID4gICAgICAgICAgICBpbm9k ZSBjYWNoZSBzaHJpbmtlcnMuCj4gCj4gTmljZSBkZXNjcmlwdGlvbiwgYnV0Li4uICBpdCBuZXZl ciBhY3R1YWxseSB0b2xkIHVzIHdoYXQgdGhlIGJlbmVmaXQgb2YKPiB0aGUgY2hhbmdlcyBhcmUu ICBQcmVzdW1hYmx5IHNvbWUgdW5kZXNjcmliZWQgd29ya2xvYWQgaGFkIHNvbWUKPiB1bmRlc2Ny aWJlZCB1c2VyLXZpc2libGUgcHJvYmxlbS4gIEJ1dCB3aGF0IHdhcyB0aGF0IHdvcmtsb2FkLCBh bmQgd2hhdAo+IHdhcyB0aGUgdXNlci12aXNpYmxlIHByb2JsZW0sIGFuZCBob3cgZG9lcyB0aGUg cGF0Y2ggYWZmZWN0IGFsbCB0aGlzPwoKRm9yIFVCSUZTIGl0IHd3aWxsIGdpdmUgYSBiZW5lZml0 IGluIHRlcm1zIG9mIHNpbXBsZXIgVUJJRlMgY29kZSAtIHdlCm5vdyBoYXZlIHRvIGtlZXAgb3Vy IG93biBsaXN0IG9mIFVCSUZTIHN1cGVyYmxvY2tzLCBwcm92aWRlIGxvY2tpbmcgZm9yCml0LCBh bmQgbWFpbnRhaW4uIFRoaXMgaXMganVzdCBleHRyYSBidXJkZW4uIFNvIHRoZSBpdGVtIDIgYWJv dmUgd2lsbCBiZQp1c2VmdWwgZm9yIFVCSUZTLgoKLS0gCkJlc3QgUmVnYXJkcywKQXJ0ZW0gQml0 eXV0c2tpeSAo0JDRgNGC0ZHQvCDQkdC40YLRjtGG0LrQuNC5KQoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KeGZzIG1haWxpbmcgbGlzdAp4ZnNAb3NzLnNn aS5jb20KaHR0cDovL29zcy5zZ2kuY29tL21haWxtYW4vbGlzdGluZm8veGZzCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755961Ab0E1Hov (ORCPT ); Fri, 28 May 2010 03:44:51 -0400 Received: from smtp.nokia.com ([192.100.122.233]:39808 "EHLO mgw-mx06.nokia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754162Ab0E1Hos (ORCPT ); Fri, 28 May 2010 03:44:48 -0400 Subject: Re: [PATCH 0/5] Per superblock shrinkers V2 From: Artem Bityutskiy Reply-To: dedekind1@gmail.com To: Andrew Morton Cc: Dave Chinner , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, xfs@oss.sgi.com In-Reply-To: <20100527133223.efa4740a.akpm@linux-foundation.org> References: <1274777588-21494-1-git-send-email-david@fromorbit.com> <20100527133223.efa4740a.akpm@linux-foundation.org> Content-Type: text/plain; charset="UTF-8" Date: Fri, 28 May 2010 10:42:06 +0300 Message-ID: <1275032526.15516.83.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 (2.28.3-1.fc12) Content-Transfer-Encoding: 8bit X-OriginalArrivalTime: 28 May 2010 07:43:50.0812 (UTC) FILETIME=[839C25C0:01CAFE39] X-Nokia-AV: Clean Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2010-05-27 at 13:32 -0700, Andrew Morton wrote: > On Tue, 25 May 2010 18:53:03 +1000 > Dave Chinner wrote: > > > This series reworks the filesystem shrinkers. We currently have a > > set of issues with the current filesystem shrinkers: > > > > 1. There is an dependency between dentry and inode cache > > shrinking that is only implicitly defined by the order of > > shrinker registration. > > 2. The shrinkers need to walk the superblock list and pin > > the superblock to avoid unmount races with the sb going > > away. > > 3. The dentry cache uses per-superblock LRUs and proportions > > reclaim between all the superblocks which means we are > > doing breadth based reclaim. This means we touch every > > superblock for every shrinker call, and may only reclaim > > a single dentry at a time from a given superblock. > > 4. The inode cache has a global LRU, so it has different > > reclaim patterns to the dentry cache, despite the fact > > that the dentry cache is generally the only thing that > > pins inodes in memory. > > 5. Filesystems need to register their own shrinkers for > > caches and can't co-ordinate them with the dentry and > > inode cache shrinkers. > > Nice description, but... it never actually told us what the benefit of > the changes are. Presumably some undescribed workload had some > undescribed user-visible problem. But what was that workload, and what > was the user-visible problem, and how does the patch affect all this? For UBIFS it wwill give a benefit in terms of simpler UBIFS code - we now have to keep our own list of UBIFS superblocks, provide locking for it, and maintain. This is just extra burden. So the item 2 above will be useful for UBIFS. -- Best Regards, Artem Bityutskiy (Артём Битюцкий) From mboxrd@z Thu Jan 1 00:00:00 1970 From: Artem Bityutskiy Subject: Re: [PATCH 0/5] Per superblock shrinkers V2 Date: Fri, 28 May 2010 10:42:06 +0300 Message-ID: <1275032526.15516.83.camel@localhost> References: <1274777588-21494-1-git-send-email-david@fromorbit.com> <20100527133223.efa4740a.akpm@linux-foundation.org> Reply-To: dedekind1@gmail.com Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Cc: Dave Chinner , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, xfs@oss.sgi.com To: Andrew Morton Return-path: In-Reply-To: <20100527133223.efa4740a.akpm@linux-foundation.org> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Thu, 2010-05-27 at 13:32 -0700, Andrew Morton wrote: > On Tue, 25 May 2010 18:53:03 +1000 > Dave Chinner wrote: >=20 > > This series reworks the filesystem shrinkers. We currently have a > > set of issues with the current filesystem shrinkers: > >=20 > > 1. There is an dependency between dentry and inode cache > > shrinking that is only implicitly defined by the order of > > shrinker registration. > > 2. The shrinkers need to walk the superblock list and pin > > the superblock to avoid unmount races with the sb going > > away. > > 3. The dentry cache uses per-superblock LRUs and proportions > > reclaim between all the superblocks which means we are > > doing breadth based reclaim. This means we touch every > > superblock for every shrinker call, and may only reclaim > > a single dentry at a time from a given superblock. > > 4. The inode cache has a global LRU, so it has different > > reclaim patterns to the dentry cache, despite the fact > > that the dentry cache is generally the only thing that > > pins inodes in memory. > > 5. Filesystems need to register their own shrinkers for > > caches and can't co-ordinate them with the dentry and > > inode cache shrinkers. >=20 > Nice description, but... it never actually told us what the benefit of > the changes are. Presumably some undescribed workload had some > undescribed user-visible problem. But what was that workload, and what > was the user-visible problem, and how does the patch affect all this? For UBIFS it wwill give a benefit in terms of simpler UBIFS code - we now have to keep our own list of UBIFS superblocks, provide locking for it, and maintain. This is just extra burden. So the item 2 above will be useful for UBIFS. --=20 Best Regards, Artem Bityutskiy (=D0=90=D1=80=D1=82=D1=91=D0=BC =D0=91=D0=B8=D1=82=D1=8E= =D1=86=D0=BA=D0=B8=D0=B9) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail202.messagelabs.com (mail202.messagelabs.com [216.82.254.227]) by kanga.kvack.org (Postfix) with ESMTP id 974516B01B6 for ; Fri, 28 May 2010 03:44:38 -0400 (EDT) Subject: Re: [PATCH 0/5] Per superblock shrinkers V2 From: Artem Bityutskiy Reply-To: dedekind1@gmail.com In-Reply-To: <20100527133223.efa4740a.akpm@linux-foundation.org> References: <1274777588-21494-1-git-send-email-david@fromorbit.com> <20100527133223.efa4740a.akpm@linux-foundation.org> Content-Type: text/plain; charset="UTF-8" Date: Fri, 28 May 2010 10:42:06 +0300 Message-ID: <1275032526.15516.83.camel@localhost> Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org To: Andrew Morton Cc: Dave Chinner , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, xfs@oss.sgi.com List-ID: On Thu, 2010-05-27 at 13:32 -0700, Andrew Morton wrote: > On Tue, 25 May 2010 18:53:03 +1000 > Dave Chinner wrote: > > > This series reworks the filesystem shrinkers. We currently have a > > set of issues with the current filesystem shrinkers: > > > > 1. There is an dependency between dentry and inode cache > > shrinking that is only implicitly defined by the order of > > shrinker registration. > > 2. The shrinkers need to walk the superblock list and pin > > the superblock to avoid unmount races with the sb going > > away. > > 3. The dentry cache uses per-superblock LRUs and proportions > > reclaim between all the superblocks which means we are > > doing breadth based reclaim. This means we touch every > > superblock for every shrinker call, and may only reclaim > > a single dentry at a time from a given superblock. > > 4. The inode cache has a global LRU, so it has different > > reclaim patterns to the dentry cache, despite the fact > > that the dentry cache is generally the only thing that > > pins inodes in memory. > > 5. Filesystems need to register their own shrinkers for > > caches and can't co-ordinate them with the dentry and > > inode cache shrinkers. > > Nice description, but... it never actually told us what the benefit of > the changes are. Presumably some undescribed workload had some > undescribed user-visible problem. But what was that workload, and what > was the user-visible problem, and how does the patch affect all this? For UBIFS it wwill give a benefit in terms of simpler UBIFS code - we now have to keep our own list of UBIFS superblocks, provide locking for it, and maintain. This is just extra burden. So the item 2 above will be useful for UBIFS. -- Best Regards, Artem Bityutskiy (D?N?N?N?D 1/4 D?D,N?N?N?DoD,D1) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org