From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id p689t2uC231696 for ; Fri, 8 Jul 2011 04:55:03 -0500 Received: from ipmail05.adl6.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 51BB0E9433D for ; Fri, 8 Jul 2011 02:55:00 -0700 (PDT) Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id NZPog5wDupvYkE1x for ; Fri, 08 Jul 2011 02:55:00 -0700 (PDT) Date: Fri, 8 Jul 2011 19:54:56 +1000 From: Dave Chinner Subject: Re: [PATCH 03/27] xfs: use write_cache_pages for writeback clustering Message-ID: <20110708095456.GI1026@dastard> References: <20110629140109.003209430@bombadil.infradead.org> <20110629140336.950805096@bombadil.infradead.org> <20110701022248.GM561@dastard> <20110701041851.GN561@dastard> <20110701093305.GA28531@infradead.org> <20110701154136.GA17881@localhost> <20110704032534.GD1026@dastard> <20110706151229.GA1998@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20110706151229.GA1998@redhat.com> List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: xfs-bounces@oss.sgi.com Errors-To: xfs-bounces@oss.sgi.com To: Johannes Weiner Cc: Christoph Hellwig , "linux-mm@kvack.org" , Wu Fengguang , Mel Gorman , "xfs@oss.sgi.com" T24gV2VkLCBKdWwgMDYsIDIwMTEgYXQgMDU6MTI6MjlQTSArMDIwMCwgSm9oYW5uZXMgV2VpbmVy IHdyb3RlOgo+IE9uIE1vbiwgSnVsIDA0LCAyMDExIGF0IDAxOjI1OjM0UE0gKzEwMDAsIERhdmUg Q2hpbm5lciB3cm90ZToKPiA+IE9uIEZyaSwgSnVsIDAxLCAyMDExIGF0IDExOjQxOjM2UE0gKzA4 MDAsIFd1IEZlbmdndWFuZyB3cm90ZToKPiA+IFdlIGhhdmUgdG8gcmVtZW1iZXIgdGhhdCBtZW1v cnkgcmVjbGFpbSBpcyBkb2luZyBMUlUgcmVjbGFpbSBhbmQgdGhlCj4gPiBmbHVzaGVyIHRocmVh ZHMgYXJlIGRvaW5nICJvbGRlc3QgZmlyc3QiIHdyaXRlYmFjay4gSU9XcywgYm90aCBhcmUgdHJ5 aW5nCj4gPiB0byBvcGVyYXRlIGluIHRoZSBzYW1lIGRpcmVjdGlvbiAob2xkZXN0IHRvIHlvdW5n ZXN0KSBmb3IgdGhlIHNhbWUKPiA+IHB1cnBvc2UuICBUaGUgZnVuZGFtZW50YWwgcHJvYmxlbSB0 aGF0IG9jY3VycyB3aGVuIG1lbW9yeSByZWNsYWltCj4gPiBzdGFydHMgd3JpdGluZyBwYWdlcyBi YWNrIGZyb20gdGhlIExSVSBpcyB0aGlzOgo+ID4gCj4gPiAJLSBtZW1vcnkgcmVjbGFpbSBoYXMg cnVuIGFoZWFkIG9mIElPIHdyaXRlYmFjayAtCj4gPiAKPiA+IFRoZSBMUlUgdXN1YWxseSBsb29r cyBsaWtlIHRoaXM6Cj4gPiAKPiA+IAlvbGRlc3QJCQkJCXlvdW5nZXN0Cj4gPiAJKy0tLS0tLS0t LS0tLS0tLSstLS0tLS0tLS0tLS0tLS0rLS0tLS0tLS0tLS0tLS0rCj4gPiAJY2xlYW4JCXdyaXRl YmFjawlkaXJ0eQo+ID4gCQkJXgkJXgo+ID4gCQkJfAkJfAo+ID4gCQkJfAkJV2hlcmUgZmx1c2hl ciB3aWxsIG5leHQgd29yayBmcm9tCj4gPiAJCQl8CQlXaGVyZSBrc3dhcGQgaXMgd29ya2luZyBm cm9tCj4gPiAJCQl8Cj4gPiAJCQlJTyBzdWJtaXR0ZWQgYnkgZmx1c2hlciwgd2FpdGluZyBvbiBj b21wbGV0aW9uCj4gPiAKPiA+IAo+ID4gSWYgbWVtb3J5IHJlY2xhaW0gaXMgaGl0dGluZyBkaXJ0 eSBwYWdlcyBvbiB0aGUgTFJVLCBpdCBtZWFucyBpdCBoYXMKPiA+IGdvdCBhaGVhZCBvZiB3cml0 ZWJhY2sgd2l0aG91dCBiZWluZyB0aHJvdHRsZWQgLSBpdCdzIHBhc3NlZCBvdmVyCj4gPiBhbGwg dGhlIHBhZ2VzIGN1cnJlbnRseSB1bmRlciB3cml0ZWJhY2sgYW5kIGlzIHRyeWluZyB0byB3cml0 ZSBiYWNrCj4gPiBwYWdlcyB0aGF0IGFyZSAqbmV3ZXIqIHRoYW4gd2hhdCB3cml0ZWJhY2sgaXMg d29ya2luZyBvbi4gSU9XcywgaXQKPiA+IHN0YXJ0cyB0cnlpbmcgdG8gZG8gdGhlIGpvYiBvZiB0 aGUgZmx1c2hlciB0aHJlYWRzLCBhbmQgaXQgZG9lcyB0aGF0Cj4gPiB2ZXJ5IGJhZGx5Lgo+ID4g Cj4gPiBUaGUgJDEwMCBxdWVzdGlvbiBpcyDiiJd3aHkgaXMgaXQgZ2V0dGluZyBhaGVhZCBvZiB3 cml0ZWJhY2sqPwo+IAo+IFVubGVzcyB5b3UgaGF2ZSBhIHB1cmVseSBzZXF1ZW50aWFsIHdyaXRl ciwgdGhlIExSVSBvcmRlciBpcyAtIGF0Cj4gbGVhc3QgaW4gdGhlb3J5IC0gZGl2ZXJnaW5nIGF3 YXkgZnJvbSB0aGUgd3JpdGViYWNrIG9yZGVyLgoKV2hpY2ggaXMgdGhlIHJvb3QgY2F1c2Ugb2Yg dGhlIElPIGNvbGxhcHNlIHRoYXQgd3JpdGViYWNrIGZyb20gdGhlCkxSVSBjYXVzZXMsIHllcz8K Cj4gQWNjb3JkaW5nIHRvIHRoZSByZWFzb25pbmcgYmVoaW5kIGdlbmVyYXRpb25hbCBnYXJiYWdl IGNvbGxlY3Rpb24sCj4gdGhleSBzaG91bGQgaW4gZmFjdCBiZSBpbnZlcnNlIHRvIGVhY2ggb3Ro ZXIuICBUaGUgb2xkZXN0IHBhZ2VzIHN0aWxsCj4gaW4gdXNlIGFyZSB0aGUgbW9zdCBsaWtlbHkg dG8gYmUgc3RpbGwgbmVlZGVkIGluIHRoZSBmdXR1cmUuCj4gCj4gSW4gcHJhY3RpY2Ugd2Ugb25s eSBtYWtlIGEgZ2VuZXJhdGlvbmFsIGRpc3RpbmN0aW9uIGJldHdlZW4gdXNlZC1vbmNlCj4gYW5k IHVzZWQtbWFueSwgd2hpY2ggbWFuaWZlc3RzIGluIHRoZSBpbmFjdGl2ZSBhbmQgdGhlIGFjdGl2 ZSBsaXN0Lgo+IEJ1dCBzdGlsbCwgd2hlbiByZWNsYWltIHN0YXJ0cyBvZmYgd2l0aCBhIGxvY2Fs aXplZCB3cml0ZXIsIHRoZSBvbGRlc3QKPiBwYWdlcyBhcmUgbGlrZWx5IHRvIGJlIGF0IHRoZSBl bmQgb2YgdGhlIGFjdGl2ZSBsaXN0LgoKWWV0IHRoZSBmaWxlIHBhZ2VzIG9uIHRoZSBhY3RpdmUg bGlzdCBhcmUgdW5saWtlbHkgdG8gYmUgZGlydHkgLQpvdmVyd3JpdGUtaW4tcGxhY2UgY2FjaGUg aG90IHdvcmtsb2FkcyBhcmUgcHJldHR5IHNjYXJjZSBpbiBteQpleHBlcmllbmNlLiBoZW5jZSB3 cml0ZWJhY2sgb2YgZGlydHkgcGFnZXMgZnJvbSB0aGUgYWN0aXZlIExSVSBpcwp1bmxpa2VseSB0 byBiZSBhIHByb2JsZW0uCgpUaGF0IGxlYXZlcyBhbGwgdGhlIHVzZS1vbmNlIHBhZ2VzIGN5Y2xp bmcgdGhyb3VnaCB0aGUgaW5hY3RpdmUKbGlzdC4gVGhlIG9sZGVzdCBwYWdlcyBvbiB0aGlzIGxp c3QgYXJlIHRoZSBvbmVzIHRoYXQgZ2V0IHJlY2xhaW1lZCwKYW5kIGlmIHdlIGFyZSBnZXR0aW5n IGxvdHMgb2YgZGlydHkgcGFnZXMgaGVyZSBpdCBzZWVtcyBwcmV0dHkgY2xlYXIKdGhhdCBtZW1v cnkgZGVtYW5kIGlzIG1vc3RseSBmb3IgcGFnZXMgYmVpbmcgcmFwaWRseSBkaXJ0aWVkLiBJbgp3 aGljaCBjYXNlLCB0cnlpbmcgdG8gc3BlZWQgdXAgdGhlIHJhdGUgYXQgd2hpY2ggdGhleSBhcmUg Y2xlYW5lZCBieQppc3N1aW5nIElPIGlzIG9ubHkgZWZmZWN0aXZlIGlmIHRoZXJlIGlzIG5vIElP IGFscmVhZHkgaW4gcHJvZ3Jlc3MuCgpXaG8ga25vd3MgaWYgSW8gaXMgYWxyZWFkeSBpbiBwcm9n cmVzcz8gVGhlIHdyaXRlYmFjayBzdWJzeXN0ZW0uLi4uCgo+IFNvIHBhZ2VzIGZyb20gdGhlIGlu YWN0aXZlIGxpc3QgYXJlIGxpa2VseSB0byBiZSB3cml0dGVuIGluIHRoZSByaWdodAo+IG9yZGVy LCBidXQgYXQgdGhlIHNhbWUgdGltZSBhY3RpdmUgcGFnZXMgYXJlIGV2ZW4gb2xkZXIsIHRodXMg d3JpdHRlbgo+IGJlZm9yZSB0aGVtLiAgTWVtb3J5IHJlY2xhaW0gc3RhcnRzIHdpdGggdGhlIGlu YWN0aXZlIHBhZ2VzLCBhbmQgdGhpcwo+IGlzIHdoeSBpdCBnZXRzIGFoZWFkLgoKQWxsIHJpZ2h0 LCBpZiB0aGUgZGVzaWduIGlzIHN1Y2ggdGhhdCB5b3UgY2FuJ3QgYXZvaWQgaGF2aW5nIHJlY2xh aW0Kd3JpdGUgYmFjayBkaXJ0eSBwYWdlcyBhcyBpdCBlbmNvdW50ZXJzIHRoZW0gb24gdGhlIGlu YWN0aXZlIExSVSwKc2hvdWxkIHRoZSBkaXJ0eSBwYWdlcyBldmVuIGJlIG9uIHRoYXQgTFJVPwoK VGhhdCBpcywgZGlydHkgcGFnZXMgY2Fubm90IGJlIHJlY2xhaW1lZCBpbW1lZGlhdGVseSBidXQg dGhleSBhcmUKaW50ZXJ0d2luZWQgd2l0aCBwYWdlcyB0aGF0IGNhbiBiZSByZWNsYWltZWQgaW1t ZWRpYXRlbHkuIFdlIHJlYWxseQp3YW50IHRvIHJlY2xhaW0gcGFnZXMgdGhhdCBjYW4gYmUgcmVj bGFpbWVkIHF1aWNrbHkgd2hpbGUgbm90CmJsb2NraW5nIG9uIG9yIGNvbnRpbnVhbGx5IGhhdmlu ZyB0byBza2lwIG92ZXIgcGFnZXMgdGhhdCBjYW5ub3QgYmUKcmVjbGFpbWVkLgoKU28gd2h5IG5v dCBtYWtlIGEgZGlzdGluY3Rpb24gYmV0d2VlbiBjbGVhbiBhbmQgZGlydHkgZmlsZSBwYWdlcyBv bgp0aGUgaW5hY3RpdmUgbGlzdD8gVGhhdCBpcywgY29uc2lkZXIgZGlydHkgcGFnZXMgdG8gc3Rp bGwgYmUgImluCnVzZSIgYW5kICJvd25lZCIgYnkgdGhlIHdyaXRlYmFjayBzdWJzeXN0ZW0uIHdo aWxlIHBhZ2VzIGFyZSBkaXJ0eQp0aGV5IGFyZSBrZXB0IG9uIGEgc2VwYXJhdGUgImRpcnR5IGZp bGUgcGFnZSBMUlUiIHRoYXQgbWVtb3J5CnJlY2xhaW0gZG9lcyBub3QgZXZlciB0b3VjaCB1bmxl c3MgaXQgcnVucyBvdXQgb2YgY2xlYW4gcGFnZXMgb24gdGhlCmluYWN0aXZlIGxpc3QgdG8gcmVj bGFpbS4gQW5kIHRoZW4gd2hlbiBpdCBydW5zIG91dCBvZiBjbGVhbiBwYWdlcywKaXQgY2FuIGdv IGZpbmQgcGFnZXMgdW5kZXIgd3JpdGViYWNrIG9uIHRoZSBkaXJ0eSBsaXN0IGFuZCBibG9jayBv bgp0aGVtIGJlZm9yZSBnb2luZyBiYWNrIHRvIHJlY2xhaW1pbmcgb2ZmIHRoZSBjbGVhbiBsaXN0 Li4uLgoKQW5kIGdpdmVuIHRoYXQgY2dyb3VwcyBoYXZlIHRoZWlyIG93biBMUlVzIGZvciByZWNs YWltIG5vdywgdGhpcwpwcm9ibGVtIG9mIGRpcnR5IHBhZ2VzIGJlaW5nIHdyaXR0ZW4gZnJvbSB0 aGUgTFJVcyBoYXMgYSBtdWNoIGxhcmdlcgpzY29wZS4gIEl0J3Mgbm90IGp1c3Qgd2hldGhlciB0 aGUgZ2xvYmFsIExSVSByZWNsYWltIGlzIGhpdHRpbmcKZGlydHkgcGFnZXMsIGl0J3MgYSBwZXIt Y2dyb3VwIHByb2JsZW0gYW5kIHRoZXkgYXJlIG11Y2ggbW9yZSBsaWtlbHkKdG8gaGF2ZSBsb3cg bWVtb3J5IGxpbWl0cyB0aGF0IGxlYWQgdG8gc3VjaCBwcm9ibGVtcy4gQW5kCmNvbmN1cnJlbnRs eSBhdCB0aGF0LCB0b28uICBXcml0ZWJhY2sgc2ltcGx5IGRvZXMndCBzY2FsZSB0byBoYXZpbmcK bXVsdGlwbGUgc291cmNlcyBvZiByYW5kb20gcGFnZSBJTyBiZWluZyBkZXNwYXRjaGVkIGNvbmN1 cnJlbnRseS4KCj4gVGhlbiB0aGVyZSBpcyBhbHNvIHRoZSBjYXNlIHdoZXJlIGEgZmFzdCB3cml0 ZXIgcHVzaGVzIGRpcnR5IHBhZ2VzIHRvCj4gdGhlIGVuZCBvZiB0aGUgTFJVIGxpc3QsIG9mIGNv dXJzZSwgYnV0IHlvdSBhbHJlYWR5IHNhaWQgdGhhdCB0aGlzIGlzCj4gbm90IGFwcGxpY2FibGUg dG8geW91ciB3b3JrbG9hZC4KPiAKPiBNeSBwb2ludCBpcyB0aGF0IEkgZG9uJ3QgdGhpbmsgaXQn cyB1bmV4cGVjdGVkIHRoYXQgZGlydHkgcGFnZXMgY29tZQo+IG9mZiB0aGUgaW5hY3RpdmUgbGlz dCBpbiBwcmFjdGljZS4gIEl0IGp1c3Qgc3Vja3MgaG93IHdlIGhhbmRsZSB0aGVtLgoKRXhhY3Rs eSB3aGF0IEkndmUgYmVlbiBzYXlpbmcuCgpBbmQgd2hhdCBJJ20gYWxzbyB0cnlpbmcgdG8gc2F5 IGlzIHRoZSB3YXkgdG8gZml4IHRoZSAid2UgZG8gc2hpdHR5CklPIG9uIGRpcnR5IHBhZ2VzIiBw cm9ibGVtIGlzICpub3QgdG8gZG8gSU8qLiBUaGF0J3MgLWV4YWN0bHktIHdoeQp0aGUgSU8tbGVz cyB3cml0ZSB0aHJvdHRsaW5nIGlzIGEgc2lnbmlmaWNhbnQgaW1wcm92ZW1lbnQ6IHdlJ3ZlCnR1 cm5lZCBzaGl0dHkgSU8gaW50byBnb29kIElPIGJ5ICp3YWl0aW5nIGZvciBJTyogZHVyaW5nIHRo cm90dGxpbmcKcmF0aGVyIHRoYW4gc3VibWl0dGluZyBJTy4KCkZ1bmRhbWVudGFsbHksIHNjYWxp bmcgdG8gTiBJTyB3YWl0ZXJzIGlzIGZhciBlYXNpZXIgYW5kIG1vcmUKZWZmaWNpZW50IHRoYW4g c2NhbGluZyB0byBOIElPIHN1Ym1pdHRlcnMuIEFsbCBJJ20gYXNraW5nIGlzIHRoYXQKeW91IGFw cGx5IHRoYXQgc2FtZSBwcmluY2lwbGUgdG8gbWVtb3J5IHJlY2xhaW0sIHBsZWFzZS4KCkNoZWVy cywKCkRhdmUuCi0tIApEYXZlIENoaW5uZXIKZGF2aWRAZnJvbW9yYml0LmNvbQoKX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KeGZzIG1haWxpbmcgbGlzdAp4 ZnNAb3NzLnNnaS5jb20KaHR0cDovL29zcy5zZ2kuY29tL21haWxtYW4vbGlzdGluZm8veGZzCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail6.bemta8.messagelabs.com (mail6.bemta8.messagelabs.com [216.82.243.55]) by kanga.kvack.org (Postfix) with ESMTP id A78539000C2 for ; Fri, 8 Jul 2011 05:55:02 -0400 (EDT) Date: Fri, 8 Jul 2011 19:54:56 +1000 From: Dave Chinner Subject: Re: [PATCH 03/27] xfs: use write_cache_pages for writeback clustering Message-ID: <20110708095456.GI1026@dastard> References: <20110629140109.003209430@bombadil.infradead.org> <20110629140336.950805096@bombadil.infradead.org> <20110701022248.GM561@dastard> <20110701041851.GN561@dastard> <20110701093305.GA28531@infradead.org> <20110701154136.GA17881@localhost> <20110704032534.GD1026@dastard> <20110706151229.GA1998@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20110706151229.GA1998@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Johannes Weiner Cc: Wu Fengguang , Christoph Hellwig , Mel Gorman , "xfs@oss.sgi.com" , "linux-mm@kvack.org" On Wed, Jul 06, 2011 at 05:12:29PM +0200, Johannes Weiner wrote: > On Mon, Jul 04, 2011 at 01:25:34PM +1000, Dave Chinner wrote: > > On Fri, Jul 01, 2011 at 11:41:36PM +0800, Wu Fengguang wrote: > > We have to remember that memory reclaim is doing LRU reclaim and the > > flusher threads are doing "oldest first" writeback. IOWs, both are trying > > to operate in the same direction (oldest to youngest) for the same > > purpose. The fundamental problem that occurs when memory reclaim > > starts writing pages back from the LRU is this: > > > > - memory reclaim has run ahead of IO writeback - > > > > The LRU usually looks like this: > > > > oldest youngest > > +---------------+---------------+--------------+ > > clean writeback dirty > > ^ ^ > > | | > > | Where flusher will next work from > > | Where kswapd is working from > > | > > IO submitted by flusher, waiting on completion > > > > > > If memory reclaim is hitting dirty pages on the LRU, it means it has > > got ahead of writeback without being throttled - it's passed over > > all the pages currently under writeback and is trying to write back > > pages that are *newer* than what writeback is working on. IOWs, it > > starts trying to do the job of the flusher threads, and it does that > > very badly. > > > > The $100 question is a??why is it getting ahead of writeback*? > > Unless you have a purely sequential writer, the LRU order is - at > least in theory - diverging away from the writeback order. Which is the root cause of the IO collapse that writeback from the LRU causes, yes? > According to the reasoning behind generational garbage collection, > they should in fact be inverse to each other. The oldest pages still > in use are the most likely to be still needed in the future. > > In practice we only make a generational distinction between used-once > and used-many, which manifests in the inactive and the active list. > But still, when reclaim starts off with a localized writer, the oldest > pages are likely to be at the end of the active list. Yet the file pages on the active list are unlikely to be dirty - overwrite-in-place cache hot workloads are pretty scarce in my experience. hence writeback of dirty pages from the active LRU is unlikely to be a problem. That leaves all the use-once pages cycling through the inactive list. The oldest pages on this list are the ones that get reclaimed, and if we are getting lots of dirty pages here it seems pretty clear that memory demand is mostly for pages being rapidly dirtied. In which case, trying to speed up the rate at which they are cleaned by issuing IO is only effective if there is no IO already in progress. Who knows if Io is already in progress? The writeback subsystem.... > So pages from the inactive list are likely to be written in the right > order, but at the same time active pages are even older, thus written > before them. Memory reclaim starts with the inactive pages, and this > is why it gets ahead. All right, if the design is such that you can't avoid having reclaim write back dirty pages as it encounters them on the inactive LRU, should the dirty pages even be on that LRU? That is, dirty pages cannot be reclaimed immediately but they are intertwined with pages that can be reclaimed immediately. We really want to reclaim pages that can be reclaimed quickly while not blocking on or continually having to skip over pages that cannot be reclaimed. So why not make a distinction between clean and dirty file pages on the inactive list? That is, consider dirty pages to still be "in use" and "owned" by the writeback subsystem. while pages are dirty they are kept on a separate "dirty file page LRU" that memory reclaim does not ever touch unless it runs out of clean pages on the inactive list to reclaim. And then when it runs out of clean pages, it can go find pages under writeback on the dirty list and block on them before going back to reclaiming off the clean list.... And given that cgroups have their own LRUs for reclaim now, this problem of dirty pages being written from the LRUs has a much larger scope. It's not just whether the global LRU reclaim is hitting dirty pages, it's a per-cgroup problem and they are much more likely to have low memory limits that lead to such problems. And concurrently at that, too. Writeback simply does't scale to having multiple sources of random page IO being despatched concurrently. > Then there is also the case where a fast writer pushes dirty pages to > the end of the LRU list, of course, but you already said that this is > not applicable to your workload. > > My point is that I don't think it's unexpected that dirty pages come > off the inactive list in practice. It just sucks how we handle them. Exactly what I've been saying. And what I'm also trying to say is the way to fix the "we do shitty IO on dirty pages" problem is *not to do IO*. That's -exactly- why the IO-less write throttling is a significant improvement: we've turned shitty IO into good IO by *waiting for IO* during throttling rather than submitting IO. Fundamentally, scaling to N IO waiters is far easier and more efficient than scaling to N IO submitters. All I'm asking is that you apply that same principle to memory reclaim, please. Cheers, Dave. -- Dave Chinner david@fromorbit.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org