From mboxrd@z Thu Jan 1 00:00:00 1970 From: Minchan Kim Subject: Re: [PATCH v6 02/12] mm: migrate: support non-lru movable page migration Date: Mon, 30 May 2016 10:33:27 +0900 Message-ID: <20160530013327.GA8683@bbox> References: <1463754225-31311-1-git-send-email-minchan@kernel.org> <1463754225-31311-3-git-send-email-minchan@kernel.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from lgeamrelo13.lge.com (LGEAMRELO13.lge.com [156.147.23.53]) by gabe.freedesktop.org (Postfix) with ESMTP id 1FC586E320 for ; Mon, 30 May 2016 01:32:59 +0000 (UTC) Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Vlastimil Babka Cc: Rik van Riel , Sergey Senozhatsky , Rafael Aquini , Jonathan Corbet , Hugh Dickins , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, virtualization@lists.linux-foundation.org, John Einar Reitan , linux-mm@kvack.org, Mel Gorman , Andrew Morton , Joonsoo Kim , Gioh Kim List-Id: dri-devel@lists.freedesktop.org T24gRnJpLCBNYXkgMjcsIDIwMTYgYXQgMDQ6MjY6MjFQTSArMDIwMCwgVmxhc3RpbWlsIEJhYmth IHdyb3RlOgo+IE9uIDA1LzIwLzIwMTYgMDQ6MjMgUE0sIE1pbmNoYW4gS2ltIHdyb3RlOgo+ID5X ZSBoYXZlIGFsbG93ZWQgbWlncmF0aW9uIGZvciBvbmx5IExSVSBwYWdlcyB1bnRpbCBub3cgYW5k IGl0IHdhcwo+ID5lbm91Z2ggdG8gbWFrZSBoaWdoLW9yZGVyIHBhZ2VzLiBCdXQgcmVjZW50bHks IGVtYmVkZGVkIHN5c3RlbShlLmcuLAo+ID53ZWJPUywgYW5kcm9pZCkgdXNlcyBsb3RzIG9mIG5v bi1tb3ZhYmxlIHBhZ2VzKGUuZy4sIHpyYW0sIEdQVSBtZW1vcnkpCj4gPnNvIHdlIGhhdmUgc2Vl biBzZXZlcmFsIHJlcG9ydHMgYWJvdXQgdHJvdWJsZXMgb2Ygc21hbGwgaGlnaC1vcmRlcgo+ID5h bGxvY2F0aW9uLiBGb3IgZml4aW5nIHRoZSBwcm9ibGVtLCB0aGVyZSB3ZXJlIHNldmVyYWwgZWZm b3J0cwo+ID4oZSxnLC4gZW5oYW5jZSBjb21wYWN0aW9uIGFsZ29yaXRobSwgU0xVQiBmYWxsYmFj ayB0byAwLW9yZGVyIHBhZ2UsCj4gPnJlc2VydmVkIG1lbW9yeSwgdm1hbGxvYyBhbmQgc28gb24p IGJ1dCBpZiB0aGVyZSBhcmUgbG90cyBvZgo+ID5ub24tbW92YWJsZSBwYWdlcyBpbiBzeXN0ZW0s IHRoZWlyIHNvbHV0aW9ucyBhcmUgdm9pZCBpbiB0aGUgbG9uZyBydW4uCj4gPgo+ID5TbywgdGhp cyBwYXRjaCBpcyB0byBzdXBwb3J0IGZhY2lsaXR5IHRvIGNoYW5nZSBub24tbW92YWJsZSBwYWdl cwo+ID53aXRoIG1vdmFibGUuIEZvciB0aGUgZmVhdHVyZSwgdGhpcyBwYXRjaCBpbnRyb2R1Y2Vz IGZ1bmN0aW9ucyByZWxhdGVkCj4gPnRvIG1pZ3JhdGlvbiB0byBhZGRyZXNzX3NwYWNlX29wZXJh dGlvbnMgYXMgd2VsbCBhcyBzb21lIHBhZ2UgZmxhZ3MuCj4gPgo+ID5JZiBhIGRyaXZlciB3YW50 IHRvIG1ha2Ugb3duIHBhZ2VzIG1vdmFibGUsIGl0IHNob3VsZCBkZWZpbmUgdGhyZWUgZnVuY3Rp b25zCj4gPndoaWNoIGFyZSBmdW5jdGlvbiBwb2ludGVycyBvZiBzdHJ1Y3QgYWRkcmVzc19zcGFj ZV9vcGVyYXRpb25zLgo+ID4KPiA+MS4gYm9vbCAoKmlzb2xhdGVfcGFnZSkgKHN0cnVjdCBwYWdl ICpwYWdlLCBpc29sYXRlX21vZGVfdCBtb2RlKTsKPiA+Cj4gPldoYXQgVk0gZXhwZWN0cyBvbiBp c29sYXRlX3BhZ2UgZnVuY3Rpb24gb2YgZHJpdmVyIGlzIHRvIHJldHVybiAqdHJ1ZSoKPiA+aWYg ZHJpdmVyIGlzb2xhdGVzIHBhZ2Ugc3VjY2Vzc2Z1bGx5LiBPbiByZXR1cmluZyB0cnVlLCBWTSBt YXJrcyB0aGUgcGFnZQo+ID5hcyBQR19pc29sYXRlZCBzbyBjb25jdXJyZW50IGlzb2xhdGlvbiBp biBzZXZlcmFsIENQVXMgc2tpcCB0aGUgcGFnZQo+ID5mb3IgaXNvbGF0aW9uLiBJZiBhIGRyaXZl ciBjYW5ub3QgaXNvbGF0ZSB0aGUgcGFnZSwgaXQgc2hvdWxkIHJldHVybiAqZmFsc2UqLgo+ID4K PiA+T25jZSBwYWdlIGlzIHN1Y2Nlc3NmdWxseSBpc29sYXRlZCwgVk0gdXNlcyBwYWdlLmxydSBm aWVsZHMgc28gZHJpdmVyCj4gPnNob3VsZG4ndCBleHBlY3QgdG8gcHJlc2VydmUgdmFsdWVzIGlu IHRoYXQgZmllbGRzLgo+ID4KPiA+Mi4gaW50ICgqbWlncmF0ZXBhZ2UpIChzdHJ1Y3QgYWRkcmVz c19zcGFjZSAqbWFwcGluZywKPiA+CQlzdHJ1Y3QgcGFnZSAqbmV3cGFnZSwgc3RydWN0IHBhZ2Ug Km9sZHBhZ2UsIGVudW0gbWlncmF0ZV9tb2RlKTsKPiA+Cj4gPkFmdGVyIGlzb2xhdGlvbiwgVk0g Y2FsbHMgbWlncmF0ZXBhZ2Ugb2YgZHJpdmVyIHdpdGggaXNvbGF0ZWQgcGFnZS4KPiA+VGhlIGZ1 bmN0aW9uIG9mIG1pZ3JhdGVwYWdlIGlzIHRvIG1vdmUgY29udGVudCBvZiB0aGUgb2xkIHBhZ2Ug dG8gbmV3IHBhZ2UKPiA+YW5kIHNldCB1cCBmaWVsZHMgb2Ygc3RydWN0IHBhZ2UgbmV3cGFnZS4g S2VlcCBpbiBtaW5kIHRoYXQgeW91IHNob3VsZAo+ID5jbGVhciBQR19tb3ZhYmxlIG9mIG9sZHBh Z2UgdmlhIF9fQ2xlYXJQYWdlTW92YWJsZSB1bmRlciBwYWdlX2xvY2sgaWYgeW91Cj4gPm1pZ3Jh dGVkIHRoZSBvbGRwYWdlIHN1Y2Nlc3NmdWxseSBhbmQgcmV0dXJucyAwLgo+ID5JZiBkcml2ZXIg Y2Fubm90IG1pZ3JhdGUgdGhlIHBhZ2UgYXQgdGhlIG1vbWVudCwgZHJpdmVyIGNhbiByZXR1cm4g LUVBR0FJTi4KPiA+T24gLUVBR0FJTiwgVk0gd2lsbCByZXRyeSBwYWdlIG1pZ3JhdGlvbiBpbiBh IHNob3J0IHRpbWUgYmVjYXVzZSBWTSBpbnRlcnByZXRzCj4gPi1FQUdBSU4gYXMgInRlbXBvcmFs IG1pZ3JhdGlvbiBmYWlsdXJlIi4gT24gcmV0dXJuaW5nIGFueSBlcnJvciBleGNlcHQgLUVBR0FJ TiwKPiA+Vk0gd2lsbCBnaXZlIHVwIHRoZSBwYWdlIG1pZ3JhdGlvbiB3aXRob3V0IHJldHJ5aW5n IGluIHRoaXMgdGltZS4KPiA+Cj4gPkRyaXZlciBzaG91bGRuJ3QgdG91Y2ggcGFnZS5scnUgZmll bGQgVk0gdXNpbmcgaW4gdGhlIGZ1bmN0aW9ucy4KPiA+Cj4gPjMuIHZvaWQgKCpwdXRiYWNrX3Bh Z2UpKHN0cnVjdCBwYWdlICopOwo+ID4KPiA+SWYgbWlncmF0aW9uIGZhaWxzIG9uIGlzb2xhdGVk IHBhZ2UsIFZNIHNob3VsZCByZXR1cm4gdGhlIGlzb2xhdGVkIHBhZ2UKPiA+dG8gdGhlIGRyaXZl ciBzbyBWTSBjYWxscyBkcml2ZXIncyBwdXRiYWNrX3BhZ2Ugd2l0aCBtaWdyYXRpb24gZmFpbGVk IHBhZ2UuCj4gPkluIHRoaXMgZnVuY3Rpb24sIGRyaXZlciBzaG91bGQgcHV0IHRoZSBpc29sYXRl ZCBwYWdlIGJhY2sgdG8gdGhlIG93biBkYXRhCj4gPnN0cnVjdHVyZS4KPiA+Cj4gPjQuIG5vbi1s cnUgbW92YWJsZSBwYWdlIGZsYWdzCj4gPgo+ID5UaGVyZSBhcmUgdHdvIHBhZ2UgZmxhZ3MgZm9y IHN1cHBvcnRpbmcgbm9uLWxydSBtb3ZhYmxlIHBhZ2UuCj4gPgo+ID4qIFBHX21vdmFibGUKPiA+ Cj4gPkRyaXZlciBzaG91bGQgdXNlIHRoZSBiZWxvdyBmdW5jdGlvbiB0byBtYWtlIHBhZ2UgbW92 YWJsZSB1bmRlciBwYWdlX2xvY2suCj4gPgo+ID4Jdm9pZCBfX1NldFBhZ2VNb3ZhYmxlKHN0cnVj dCBwYWdlICpwYWdlLCBzdHJ1Y3QgYWRkcmVzc19zcGFjZSAqbWFwcGluZykKPiA+Cj4gPkl0IG5l ZWRzIGFyZ3VtZW50IG9mIGFkZHJlc3Nfc3BhY2UgZm9yIHJlZ2lzdGVyaW5nIG1pZ3JhdGlvbiBm YW1pbHkgZnVuY3Rpb25zCj4gPndoaWNoIHdpbGwgYmUgY2FsbGVkIGJ5IFZNLiBFeGFjdGx5IHNw ZWFraW5nLCBQR19tb3ZhYmxlIGlzIG5vdCBhIHJlYWwgZmxhZyBvZgo+ID5zdHJ1Y3QgcGFnZS4g UmF0aGVyIHRoYW4sIFZNIHJldXNlcyBwYWdlLT5tYXBwaW5nJ3MgbG93ZXIgYml0cyB0byByZXBy ZXNlbnQgaXQuCj4gPgo+ID4JI2RlZmluZSBQQUdFX01BUFBJTkdfTU9WQUJMRSAweDIKPiA+CXBh Z2UtPm1hcHBpbmcgPSBwYWdlLT5tYXBwaW5nIHwgUEFHRV9NQVBQSU5HX01PVkFCTEU7Cj4gCj4g SW50ZXJlc3RpbmcsIGxldCdzIHNlZSBob3cgdGhhdCB3b3JrcyBvdXQuLi4KPiAKPiBPdmVyYWwg dGhpcyBsb29rcyBtdWNoIGJldHRlciB0aGFuIHRoZSBsYXN0IHZlcnNpb24gSSBjaGVja2VkIQoK VGhhbmtzLgoKPiAKPiBbLi4uXQo+IAo+ID5AQCAtMzU3LDI5ICszNjAsMzcgQEAgUEFHRUZMQUco SWRsZSwgaWRsZSwgUEZfQU5ZKQo+ID4gICogd2l0aCB0aGUgUEFHRV9NQVBQSU5HX0FOT04gYml0 IHNldCB0byBkaXN0aW5ndWlzaCBpdC4gIFNlZSBybWFwLmguCj4gPiAgKgo+ID4gICogT24gYW4g YW5vbnltb3VzIHBhZ2UgaW4gYSBWTV9NRVJHRUFCTEUgYXJlYSwgaWYgQ09ORklHX0tTTSBpcyBl bmFibGVkLAo+ID4tICogdGhlIFBBR0VfTUFQUElOR19LU00gYml0IG1heSBiZSBzZXQgYWxvbmcg d2l0aCB0aGUgUEFHRV9NQVBQSU5HX0FOT04gYml0Owo+ID4tICogYW5kIHRoZW4gcGFnZS0+bWFw cGluZyBwb2ludHMsIG5vdCB0byBhbiBhbm9uX3ZtYSwgYnV0IHRvIGEgcHJpdmF0ZQo+ID4rICog dGhlIFBBR0VfTUFQUElOR19NT1ZBQkxFIGJpdCBtYXkgYmUgc2V0IGFsb25nIHdpdGggdGhlIFBB R0VfTUFQUElOR19BTk9OCj4gPisgKiBiaXQ7IGFuZCB0aGVuIHBhZ2UtPm1hcHBpbmcgcG9pbnRz LCBub3QgdG8gYW4gYW5vbl92bWEsIGJ1dCB0byBhIHByaXZhdGUKPiA+ICAqIHN0cnVjdHVyZSB3 aGljaCBLU00gYXNzb2NpYXRlcyB3aXRoIHRoYXQgbWVyZ2VkIHBhZ2UuICBTZWUga3NtLmguCj4g PiAgKgo+ID4tICogUEFHRV9NQVBQSU5HX0tTTSB3aXRob3V0IFBBR0VfTUFQUElOR19BTk9OIGlz IGN1cnJlbnRseSBuZXZlciB1c2VkLgo+ID4rICogUEFHRV9NQVBQSU5HX0tTTSB3aXRob3V0IFBB R0VfTUFQUElOR19BTk9OIGlzIHVzZWQgZm9yIG5vbi1scnUgbW92YWJsZQo+ID4rICogcGFnZSBh bmQgdGhlbiBwYWdlLT5tYXBwaW5nIHBvaW50cyBhIHN0cnVjdCBhZGRyZXNzX3NwYWNlLgo+ID4g ICoKPiA+ICAqIFBsZWFzZSBub3RlIHRoYXQsIGNvbmZ1c2luZ2x5LCAicGFnZV9tYXBwaW5nIiBy ZWZlcnMgdG8gdGhlIGlub2RlCj4gPiAgKiBhZGRyZXNzX3NwYWNlIHdoaWNoIG1hcHMgdGhlIHBh Z2UgZnJvbSBkaXNrOyB3aGVyZWFzICJwYWdlX21hcHBlZCIKPiA+ICAqIHJlZmVycyB0byB1c2Vy IHZpcnR1YWwgYWRkcmVzcyBzcGFjZSBpbnRvIHdoaWNoIHRoZSBwYWdlIGlzIG1hcHBlZC4KPiA+ ICAqLwo+ID4tI2RlZmluZSBQQUdFX01BUFBJTkdfQU5PTgkxCj4gPi0jZGVmaW5lIFBBR0VfTUFQ UElOR19LU00JMgo+ID4tI2RlZmluZSBQQUdFX01BUFBJTkdfRkxBR1MJKFBBR0VfTUFQUElOR19B Tk9OIHwgUEFHRV9NQVBQSU5HX0tTTSkKPiA+KyNkZWZpbmUgUEFHRV9NQVBQSU5HX0FOT04JMHgx Cj4gPisjZGVmaW5lIFBBR0VfTUFQUElOR19NT1ZBQkxFCTB4Mgo+ID4rI2RlZmluZSBQQUdFX01B UFBJTkdfS1NNCShQQUdFX01BUFBJTkdfQU5PTiB8IFBBR0VfTUFQUElOR19NT1ZBQkxFKQo+ID4r I2RlZmluZSBQQUdFX01BUFBJTkdfRkxBR1MJKFBBR0VfTUFQUElOR19BTk9OIHwgUEFHRV9NQVBQ SU5HX01PVkFCTEUpCj4gPgo+ID4tc3RhdGljIF9fYWx3YXlzX2lubGluZSBpbnQgUGFnZUFub25I ZWFkKHN0cnVjdCBwYWdlICpwYWdlKQo+ID4rc3RhdGljIF9fYWx3YXlzX2lubGluZSBpbnQgUGFn ZU1hcHBpbmdGbGFnKHN0cnVjdCBwYWdlICpwYWdlKQo+IAo+IFBhZ2VNYXBwaW5nRmxhZ3MoKT8K ClllYi4KCj4gCj4gWy4uLl0KPiAKPiA+ZGlmZiAtLWdpdCBhL21tL2NvbXBhY3Rpb24uYyBiL21t L2NvbXBhY3Rpb24uYwo+ID5pbmRleCAxNDI3MzY2YWQ2NzMuLjJkNjg2MmQwZGY2MCAxMDA2NDQK PiA+LS0tIGEvbW0vY29tcGFjdGlvbi5jCj4gPisrKyBiL21tL2NvbXBhY3Rpb24uYwo+ID5AQCAt ODEsNiArODEsNDEgQEAgc3RhdGljIGlubGluZSBib29sIG1pZ3JhdGVfYXN5bmNfc3VpdGFibGUo aW50IG1pZ3JhdGV0eXBlKQo+ID4KPiA+ICNpZmRlZiBDT05GSUdfQ09NUEFDVElPTgo+ID4KPiA+ K2ludCBQYWdlTW92YWJsZShzdHJ1Y3QgcGFnZSAqcGFnZSkKPiA+K3sKPiA+KwlzdHJ1Y3QgYWRk cmVzc19zcGFjZSAqbWFwcGluZzsKPiA+Kwo+ID4rCVdBUk5fT04oIVBhZ2VMb2NrZWQocGFnZSkp Owo+IAo+IFdoeSBub3QgVk1fQlVHX09OX1BBR0UgYXMgZWxzZXdoZXJlPwoKSSBoYXZlIGJhY2tw b3J0ZWQgdGhpcyBwYXRjaHNldCB0byBodWdlIG9sZCBrZXJuZWwgd2hpY2ggZG9lc24ndCBoYXZl ClZNX0JVR19PTl9QQUdFIGFuZCBmb3Jnb3QgdG8gY29ycmVjdCBpdCBiZWZvcnJlIHNlbmRpbmcg bWFpbmxpbmUuCgpUaGFua3MuCgo+IAo+ID4rCWlmICghX19QYWdlTW92YWJsZShwYWdlKSkKPiA+ KwkJZ290byBvdXQ7Cj4gCj4gSnVzdCByZXR1cm4gMC4KCk1heWJlIEkgbG92ZSBnb3RvLiBZb3Ug cmVhbGl6ZWQgbWUuIEkgd2lsbCBzcGxpdCB1cCB3aWxsIGhlci4KCj4gCj4gPisKPiA+KwltYXBw aW5nID0gcGFnZV9tYXBwaW5nKHBhZ2UpOwo+ID4rCWlmIChtYXBwaW5nICYmIG1hcHBpbmctPmFf b3BzICYmIG1hcHBpbmctPmFfb3BzLT5pc29sYXRlX3BhZ2UpCj4gPisJCXJldHVybiAxOwo+ID4r b3V0Ogo+ID4rCXJldHVybiAwOwo+ID4rfQo+ID4rRVhQT1JUX1NZTUJPTChQYWdlTW92YWJsZSk7 Cj4gPisKPiA+K3ZvaWQgX19TZXRQYWdlTW92YWJsZShzdHJ1Y3QgcGFnZSAqcGFnZSwgc3RydWN0 IGFkZHJlc3Nfc3BhY2UgKm1hcHBpbmcpCj4gPit7Cj4gPisJVk1fQlVHX09OX1BBR0UoIVBhZ2VM b2NrZWQocGFnZSksIHBhZ2UpOwo+ID4rCVZNX0JVR19PTl9QQUdFKCh1bnNpZ25lZCBsb25nKW1h cHBpbmcgJiBQQUdFX01BUFBJTkdfTU9WQUJMRSwgcGFnZSk7Cj4gPisJcGFnZS0+bWFwcGluZyA9 ICh2b2lkICopKCh1bnNpZ25lZCBsb25nKW1hcHBpbmcgfCBQQUdFX01BUFBJTkdfTU9WQUJMRSk7 Cj4gPit9Cj4gPitFWFBPUlRfU1lNQk9MKF9fU2V0UGFnZU1vdmFibGUpOwo+ID4rCj4gPit2b2lk IF9fQ2xlYXJQYWdlTW92YWJsZShzdHJ1Y3QgcGFnZSAqcGFnZSkKPiA+K3sKPiA+KwlWTV9CVUdf T05fUEFHRSghUGFnZUxvY2tlZChwYWdlKSwgcGFnZSk7Cj4gPisJVk1fQlVHX09OX1BBR0UoIVBh Z2VNb3ZhYmxlKHBhZ2UpLCBwYWdlKTsKPiA+KwlWTV9CVUdfT05fUEFHRSghKCh1bnNpZ25lZCBs b25nKXBhZ2UtPm1hcHBpbmcgJiBQQUdFX01BUFBJTkdfTU9WQUJMRSksCj4gPisJCQkJcGFnZSk7 Cj4gCj4gVGhlIGxhc3QgbGluZSBzb3VuZHMgcmVkdW5kYW50LCBQYWdlTW92YWJsZSgpIGFscmVh ZHkgY2hlY2tlZCB0aGlzCj4gdmlhIF9fUGFnZU1vdmFibGUoKQoKWWViLgoKPiAKPiAKPiA+Kwlw YWdlLT5tYXBwaW5nID0gKHZvaWQgKikoKHVuc2lnbmVkIGxvbmcpcGFnZS0+bWFwcGluZyAmCj4g PisJCQkJUEFHRV9NQVBQSU5HX01PVkFCTEUpOwo+IAo+IFRoaXMgc2hvdWxkIGJlIG5lZ2F0ZWQg dG8gY2xlYXIuLi4gdXNlIH5QQUdFX01BUFBJTkdfTU9WQUJMRSA/CgpOby4KClRoZSBpbnRlbnRp b24gaXMgdG8gY2xlYXIgb25seSBtYXBwaW5nIHZhbHVlIGJ1dCBQQUdFX01BUFBJTkdfTU9WQUJM RQpmbGFnLiBTbywgYW55IG5ldyBtaWdyYXRpb24gdHJpYWwgd2lsbCBiZSBmYWlsZWQgYmVjYXVz ZSBQYWdlTW92YWJsZQpjaGVja3MgcGFnZSdzIG1hcHBpbmcgdmFsdWUgYnV0IG9uZ29pbmcgbWln cmFpb24gaGFuZGxpbmcgY2FuIGNhdGNoCndoZXRoZXIgaXQncyBtb3ZhYmxlIHBhZ2Ugb3Igbm90 IHdpdGggdGhlIHR5cGUgYml0LgoKRm9yIGV4YW1wbGUsIHdlIG5lZWQgdG8ga2VlcCB0aGUgdHlw ZSBiaXQgdG8gaGFuZGxlIHB1dGJhY2sgdGhlIHBhZ2UuCldpdGggcGFyYWxsZWwgZnJlZWluZyhl LmcuLCBfX0NsZWFyUGFnZU1vdmFibGUpIGZyb20gdGhlIG93bmVyIGFmdGVyCmlzb2xhdGluZyB0 aGUgcGFnZSwgbWlncmF0aW9uIHdpbGwgYmUgZmFpbGVkIGFuZCBwdXQgaXQgYmFjay4gVGhlbiwK d2UgbmVlZCB0byBpZGVudGlmeSBtb3ZhYmxlIHRvIGNsZWFyIFBHX2lzb2xhdGUgYW5kIHNob3Vs ZG4ndCBjYWxsCm1hcHBpbmctPmFfb3BzLT5wdXRiYWNrX3BhZ2UuIEZvciB0aGF0LCB3ZSBuZWVk IHRvIGtlZXAgdGhlIHR5cGUgYml0CnVudGlsIHRoZSBwYWdlIGlzIHJldHVybiB0byBidWRkeS4K Cj4gCj4gPit9Cj4gPitFWFBPUlRfU1lNQk9MKF9fQ2xlYXJQYWdlTW92YWJsZSk7Cj4gPisKPiA+ IC8qIERvIG5vdCBza2lwIGNvbXBhY3Rpb24gbW9yZSB0aGFuIDY0IHRpbWVzICovCj4gPiAjZGVm aW5lIENPTVBBQ1RfTUFYX0RFRkVSX1NISUZUIDYKPiA+Cj4gPkBAIC03MzUsMjEgKzc3MCw2IEBA IGlzb2xhdGVfbWlncmF0ZXBhZ2VzX2Jsb2NrKHN0cnVjdCBjb21wYWN0X2NvbnRyb2wgKmNjLCB1 bnNpZ25lZCBsb25nIGxvd19wZm4sCj4gPiAJCX0KPiA+Cj4gPiAJCS8qCj4gPi0JCSAqIENoZWNr IG1heSBiZSBsb2NrbGVzcyBidXQgdGhhdCdzIG9rIGFzIHdlIHJlY2hlY2sgbGF0ZXIuCj4gPi0J CSAqIEl0J3MgcG9zc2libGUgdG8gbWlncmF0ZSBMUlUgcGFnZXMgYW5kIGJhbGxvb24gcGFnZXMK PiA+LQkJICogU2tpcCBhbnkgb3RoZXIgdHlwZSBvZiBwYWdlCj4gPi0JCSAqLwo+ID4tCQlpc19s cnUgPSBQYWdlTFJVKHBhZ2UpOwo+ID4tCQlpZiAoIWlzX2xydSkgewo+ID4tCQkJaWYgKHVubGlr ZWx5KGJhbGxvb25fcGFnZV9tb3ZhYmxlKHBhZ2UpKSkgewo+ID4tCQkJCWlmIChiYWxsb29uX3Bh Z2VfaXNvbGF0ZShwYWdlKSkgewo+ID4tCQkJCQkvKiBTdWNjZXNzZnVsbHkgaXNvbGF0ZWQgKi8K PiA+LQkJCQkJZ290byBpc29sYXRlX3N1Y2Nlc3M7Cj4gPi0JCQkJfQo+ID4tCQkJfQo+ID4tCQl9 Cj4gCj4gU28gdGhpcyBlZmZlY3RpdmVseSBwcmV2ZW50cyBtb3ZhYmxlIGNvbXBvdW5kIHBhZ2Vz IGZyb20gYmVpbmcKPiBtaWdyYXRlZC4gQXJlIHlvdSBzdXJlIG5vIHVzZXJzIG9mIHRoaXMgZnVu Y3Rpb25hbGl0eSBhcmUgZ29pbmcgdG8KPiBoYXZlIGNvbXBvdW5kIHBhZ2VzPyBJIGFzc3VtZWQg dGhhdCB0aGV5IGNvdWxkLCBhbmQgc28gbWFkZSB0aGUgY29kZQo+IGxpa2UgdGhpcywgd2l0aCB0 aGUgaXNfbHJ1IHZhcmlhYmxlICh3aGljaCBpcyByZWR1bmRhbnQgYWZ0ZXIgeW91cgo+IGNoYW5n ZSkuCgpUaGlzIGltcGxlbWVudGF0aW9uIGF0IHRoZSBtb21lbnQgZGlzYWJsZXMgZWZmZWN0aXZl bHkgbm9uLWxydSBjb21wb3VuZApwYWdlIG1pZ3JhdGlvbiBidXQgSSdtIG5vdCBhIGdvZCBzbyBJ IGNhbid0IG1ha2Ugc3VyZSBubyBvbmUgZG9lc24ndCB3YW50Cml0IGluIGZ1dHVyZS4gSWYgc29t ZW9uZSB3YW50IGl0LCB3ZSBjYW4gc3VwcG9ydCBpdCB0aGVuIGJlY2F1c2UgdGhpcyB3b3JrCmRv ZXNuJ3QgcHJldmVudCBpdCBieSBkZXNpZ24uCgo+IAo+ID4tCQkvKgo+ID4gCQkgKiBSZWdhcmRs ZXNzIG9mIGJlaW5nIG9uIExSVSwgY29tcG91bmQgcGFnZXMgc3VjaCBhcyBUSFAgYW5kCj4gPiAJ CSAqIGh1Z2V0bGJmcyBhcmUgbm90IHRvIGJlIGNvbXBhY3RlZC4gV2UgY2FuIHBvdGVudGlhbGx5 IHNhdmUKPiA+IAkJICogYSBsb3Qgb2YgaXRlcmF0aW9ucyBpZiB3ZSBza2lwIHRoZW0gYXQgb25j ZS4gVGhlIGNoZWNrIGlzCj4gPkBAIC03NjUsOCArNzg1LDM4IEBAIGlzb2xhdGVfbWlncmF0ZXBh Z2VzX2Jsb2NrKHN0cnVjdCBjb21wYWN0X2NvbnRyb2wgKmNjLCB1bnNpZ25lZCBsb25nIGxvd19w Zm4sCj4gPiAJCQlnb3RvIGlzb2xhdGVfZmFpbDsKPiA+IAkJfQo+ID4KPiA+LQkJaWYgKCFpc19s cnUpCj4gPisJCS8qCj4gPisJCSAqIENoZWNrIG1heSBiZSBsb2NrbGVzcyBidXQgdGhhdCdzIG9r IGFzIHdlIHJlY2hlY2sgbGF0ZXIuCj4gPisJCSAqIEl0J3MgcG9zc2libGUgdG8gbWlncmF0ZSBM UlUgYW5kIG5vbi1scnUgbW92YWJsZSBwYWdlcy4KPiA+KwkJICogU2tpcCBhbnkgb3RoZXIgdHlw ZSBvZiBwYWdlCj4gPisJCSAqLwo+ID4rCQlpc19scnUgPSBQYWdlTFJVKHBhZ2UpOwo+ID4rCQlp ZiAoIWlzX2xydSkgewo+ID4rCQkJaWYgKHVubGlrZWx5KGJhbGxvb25fcGFnZV9tb3ZhYmxlKHBh Z2UpKSkgewo+ID4rCQkJCWlmIChiYWxsb29uX3BhZ2VfaXNvbGF0ZShwYWdlKSkgewo+ID4rCQkJ CQkvKiBTdWNjZXNzZnVsbHkgaXNvbGF0ZWQgKi8KPiA+KwkJCQkJZ290byBpc29sYXRlX3N1Y2Nl c3M7Cj4gPisJCQkJfQo+ID4rCQkJfQo+IAo+IFsuLi5dCj4gCj4gPitib29sIGlzb2xhdGVfbW92 YWJsZV9wYWdlKHN0cnVjdCBwYWdlICpwYWdlLCBpc29sYXRlX21vZGVfdCBtb2RlKQo+ID4rewo+ ID4rCXN0cnVjdCBhZGRyZXNzX3NwYWNlICptYXBwaW5nOwo+ID4rCj4gPisJLyoKPiA+KwkgKiBB dm9pZCBidXJuaW5nIGN5Y2xlcyB3aXRoIHBhZ2VzIHRoYXQgYXJlIHlldCB1bmRlciBfX2ZyZWVf cGFnZXMoKSwKPiA+KwkgKiBvciBqdXN0IGdvdCBmcmVlZCB1bmRlciB1cy4KPiA+KwkgKgo+ID4r CSAqIEluIGNhc2Ugd2UgJ3dpbicgYSByYWNlIGZvciBhIG1vdmFibGUgcGFnZSBiZWluZyBmcmVl ZCB1bmRlciB1cyBhbmQKPiA+KwkgKiByYWlzZSBpdHMgcmVmY291bnQgcHJldmVudGluZyBfX2Zy ZWVfcGFnZXMoKSBmcm9tIGRvaW5nIGl0cyBqb2IKPiA+KwkgKiB0aGUgcHV0X3BhZ2UoKSBhdCB0 aGUgZW5kIG9mIHRoaXMgYmxvY2sgd2lsbCB0YWtlIGNhcmUgb2YKPiA+KwkgKiByZWxlYXNlIHRo aXMgcGFnZSwgdGh1cyBhdm9pZGluZyBhIG5hc3R5IGxlYWthZ2UuCj4gPisJICovCj4gPisJaWYg KHVubGlrZWx5KCFnZXRfcGFnZV91bmxlc3NfemVybyhwYWdlKSkpCj4gPisJCWdvdG8gb3V0Owo+ ID4rCj4gPisJLyoKPiA+KwkgKiBDaGVjayBQYWdlTW92YWJsZSBiZWZvcmUgaG9sZGluZyBhIFBH X2xvY2sgYmVjYXVzZSBwYWdlJ3Mgb3duZXIKPiA+KwkgKiBhc3N1bWVzIGFueWJvZHkgZG9lc24n dCB0b3VjaCBQR19sb2NrIG9mIG5ld2x5IGFsbG9jYXRlZCBwYWdlCj4gPisJICogc28gdW5jb25k aXRpb25hbGx5IGdyYXBwaW5nIHRoZSBsb2NrIHJ1aW5zIHBhZ2UncyBvd25lciBzaWRlLgo+ID4r CSAqLwo+ID4rCWlmICh1bmxpa2VseSghX19QYWdlTW92YWJsZShwYWdlKSkpCj4gPisJCWdvdG8g b3V0X3B1dHBhZ2U7Cj4gPisJLyoKPiA+KwkgKiBBcyBtb3ZhYmxlIHBhZ2VzIGFyZSBub3QgaXNv bGF0ZWQgZnJvbSBMUlUgbGlzdHMsIGNvbmN1cnJlbnQKPiA+KwkgKiBjb21wYWN0aW9uIHRocmVh ZHMgY2FuIHJhY2UgYWdhaW5zdCBwYWdlIG1pZ3JhdGlvbiBmdW5jdGlvbnMKPiA+KwkgKiBhcyB3 ZWxsIGFzIHJhY2UgYWdhaW5zdCB0aGUgcmVsZWFzaW5nIGEgcGFnZS4KPiA+KwkgKgo+ID4rCSAq IEluIG9yZGVyIHRvIGF2b2lkIGhhdmluZyBhbiBhbHJlYWR5IGlzb2xhdGVkIG1vdmFibGUgcGFn ZQo+ID4rCSAqIGJlaW5nICh3cm9uZ2x5KSByZS1pc29sYXRlZCB3aGlsZSBpdCBpcyB1bmRlciBt aWdyYXRpb24sCj4gPisJICogb3IgdG8gYXZvaWQgYXR0ZW1wdGluZyB0byBpc29sYXRlIHBhZ2Vz IGJlaW5nIHJlbGVhc2VkLAo+ID4rCSAqIGxldHMgYmUgc3VyZSB3ZSBoYXZlIHRoZSBwYWdlIGxv Y2sKPiA+KwkgKiBiZWZvcmUgcHJvY2VlZGluZyB3aXRoIHRoZSBtb3ZhYmxlIHBhZ2UgaXNvbGF0 aW9uIHN0ZXBzLgo+ID4rCSAqLwo+ID4rCWlmICh1bmxpa2VseSghdHJ5bG9ja19wYWdlKHBhZ2Up KSkKPiA+KwkJZ290byBvdXRfcHV0cGFnZTsKPiA+Kwo+ID4rCWlmICghUGFnZU1vdmFibGUocGFn ZSkgfHwgUGFnZUlzb2xhdGVkKHBhZ2UpKQo+ID4rCQlnb3RvIG91dF9ub19pc29sYXRlZDsKPiA+ Kwo+ID4rCW1hcHBpbmcgPSBwYWdlX21hcHBpbmcocGFnZSk7Cj4gCj4gSG1tIHNvIG9uIGZpcnN0 IHRhaWwgcGFnZSBvZiBhIFRIUCBjb21wb3VuZCBwYWdlLCBwYWdlLT5tYXBwaW5nIHdpbGwKPiBh bGlhcyB3aXRoIGNvbXBvdW5kX21hcGNvdW50LiBUaGF0IGNhbiBlYXNpbHkgaGF2ZSBhIHZhbHVl IG1hdGNoaW5nCj4gUGFnZU1vdmFibGUgZmxhZ3MgYW5kIHdlJ2xsIHByb2NlZWQgYW5kIHN0YXJ0 IGluc3BlY3RpbmcgdGhlCj4gY29tcG91bmQgaGVhZCBpbiBwYWdlX21hcHBpbmcoKS4uLiBtYXli ZSBpdCdzIG5vdCBhIGJpZyBkZWFsLCBvciB3ZQo+IGJldHRlciBjaGVjayBhbmQgc2tpcCBQYWdl VGFpbCBmaXJzdCwgbXVzdCB0aGluayBhYm91dCBpdCBtb3JlLi4uCgpJIHRob3VodCBQYWdlQ29t cG91bmQgY2hlY2sgcmlnaHQgYmVmb3JlIGlzb2xhdGVfbW92YWJsZV9wYWdlIGluCmlzb2xhdGVf bWlncmF0ZXBhZ2VzX2Jsb2NrIHdpbGwgZmlsdGVyIGl0IG91dCBtb3N0bHkgYnV0IHllYWgKaXQg aXMgcmFjeSB3aXRob3V0IHpvbmUtPmxydV9sb2NrIHNvIGl0IGNvdWxkIHJlYWNoIHRvIGlzb2xh dGVfbW92YWJsZV9wYWdlLgpIb3dldmVyLCBQYWdlTW92YWJsZSBjaGVjayBpbiB0aGVyZSBpbnZl c3RpZ2F0ZXMgbWFwcGluZywgbWFwcGluZy0+YV9vcHMsCmFuZCBhX29wcy0+aXNvbGF0ZV9wYWdl IHRvIHZlcmlmeSB3aGV0aGVyIGl0J3MgbW92YWJsZSBwYWdlIG9yIG5vdC4KCkkgdGhvdWdodCBp dCdzIHN1ZmZpY2llbnQgdG8gZmlsdGVyIFRIUCBwYWdlLgoKPiAKPiBbLi4uXQo+IAo+ID5AQCAt NzU1LDMzICs4NDQsNjkgQEAgc3RhdGljIGludCBtb3ZlX3RvX25ld19wYWdlKHN0cnVjdCBwYWdl ICpuZXdwYWdlLCBzdHJ1Y3QgcGFnZSAqcGFnZSwKPiA+IAkJCQllbnVtIG1pZ3JhdGVfbW9kZSBt b2RlKQo+ID4gewo+ID4gCXN0cnVjdCBhZGRyZXNzX3NwYWNlICptYXBwaW5nOwo+ID4tCWludCBy YzsKPiA+KwlpbnQgcmMgPSAtRUFHQUlOOwo+ID4rCWJvb2wgaXNfbHJ1ID0gIV9fUGFnZU1vdmFi bGUocGFnZSk7Cj4gPgo+ID4gCVZNX0JVR19PTl9QQUdFKCFQYWdlTG9ja2VkKHBhZ2UpLCBwYWdl KTsKPiA+IAlWTV9CVUdfT05fUEFHRSghUGFnZUxvY2tlZChuZXdwYWdlKSwgbmV3cGFnZSk7Cj4g Pgo+ID4gCW1hcHBpbmcgPSBwYWdlX21hcHBpbmcocGFnZSk7Cj4gPi0JaWYgKCFtYXBwaW5nKQo+ ID4tCQlyYyA9IG1pZ3JhdGVfcGFnZShtYXBwaW5nLCBuZXdwYWdlLCBwYWdlLCBtb2RlKTsKPiA+ LQllbHNlIGlmIChtYXBwaW5nLT5hX29wcy0+bWlncmF0ZXBhZ2UpCj4gPi0JCS8qCj4gPi0JCSAq IE1vc3QgcGFnZXMgaGF2ZSBhIG1hcHBpbmcgYW5kIG1vc3QgZmlsZXN5c3RlbXMgcHJvdmlkZSBh Cj4gPi0JCSAqIG1pZ3JhdGVwYWdlIGNhbGxiYWNrLiBBbm9ueW1vdXMgcGFnZXMgYXJlIHBhcnQg b2Ygc3dhcAo+ID4tCQkgKiBzcGFjZSB3aGljaCBhbHNvIGhhcyBpdHMgb3duIG1pZ3JhdGVwYWdl IGNhbGxiYWNrLiBUaGlzCj4gPi0JCSAqIGlzIHRoZSBtb3N0IGNvbW1vbiBwYXRoIGZvciBwYWdl IG1pZ3JhdGlvbi4KPiA+LQkJICovCj4gPi0JCXJjID0gbWFwcGluZy0+YV9vcHMtPm1pZ3JhdGVw YWdlKG1hcHBpbmcsIG5ld3BhZ2UsIHBhZ2UsIG1vZGUpOwo+ID4tCWVsc2UKPiA+LQkJcmMgPSBm YWxsYmFja19taWdyYXRlX3BhZ2UobWFwcGluZywgbmV3cGFnZSwgcGFnZSwgbW9kZSk7Cj4gPisJ LyoKPiA+KwkgKiBJbiBjYXNlIG9mIG5vbi1scnUgcGFnZSwgaXQgY291bGQgYmUgcmVsZWFzZWQg YWZ0ZXIKPiA+KwkgKiBpc29sYXRpb24gc3RlcC4gSW4gdGhhdCBjYXNlLCB3ZSBzaG91bGRuJ3Qg dHJ5Cj4gPisJICogZmFsbGJhY2sgbWlncmF0aW9uIHdoaWNoIGlzIGRlc2lnbmVkIGZvciBMUlUg cGFnZXMuCj4gPisJICovCj4gCj4gSG1tIGJ1dCBpc19scnUgd2FzIGRldGVybWluZWQgZnJvbSAh X19QYWdlTW92YWJsZSgpIGFib3ZlLCBhbHNvIHdlbGwKPiBhZnRlciB0aGUgaXNvbGF0aW9uIHN0 ZXAuIFNvIGlmIHRoZSBkcml2ZXIgYWxyZWFkeSByZWxlYXNlZCBpdCwgd2UKPiB3b3VsZG4ndCBk ZXRlY3QgaXQ/IEFuZCB0aGlzIGZ1bmN0aW9uIGlzIGFsbCB1bmRlciBzYW1lIHBhZ2UgbG9jaywK PiBzbyBpZiBfX1BhZ2VNb3ZhYmxlIHdhcyB0cnVlIGFib3ZlLCBzbyB3aWxsIGJlIFBhZ2VNb3Zh YmxlIGJlbG93PwoKWW91IGFyZSBtaXNzaW5nIHdoYXQgSSBtZW50aW9uZWQgYWJvdmUuCldlIHNo b3VsZCBrZWVwIHRoZSB0eXBlIGJpdCB0byBjYXRjaCB3aGF0IHlvdSBhcmUgc2F5aW5nKGkuZS4s IGRyaXZlcgphbHJlYWR5IHJlbGVhc2VkKS4KCl9fUGFnZU1vdmFibGUganVzdCBjaGVja3MgUEFH RV9NQVBQSU5HX01PVkFCTEUgZmxhZyBhbmQgUGFnZU1vdmFibGUKY2hlY2tzIHBhZ2UtPm1hcHBp bmcgdmFsaWQgd2hpbGUgX19DbGVhclBhZ2VNb3ZhYmxlIHJlc2V0IG9ubHkKdmFsaWQgdmF1bGUg b2YgbWFwcGluZywgbm90IFBBR0VfTUFQUElOR19NT1ZBQkxFIGZsYWcuCgpJIHdyb3RlIGl0IGRv d24gaW4gRG9jdW1lbnRhdGlvbi92bS9wYWdlX21pZ3JhdGlvbi4KCiJGb3IgdGVzdGluZyBvZiBu b24tbHJ1IG1vdmFibGUgcGFnZSwgVk0gc3VwcG9ydHMgX19QYWdlTW92YWJsZSBmdW5jdGlvbi4K SG93ZXZlciwgaXQgZG9lc24ndCBndWFyYW50ZWUgdG8gaWRlbnRpZnkgbm9uLWxydSBtb3ZhYmxl IHBhZ2UgYmVjYXVzZQpwYWdlLT5tYXBwaW5nIGZpZWxkIGlzIHVuaWZpZWQgd2l0aCBvdGhlciB2 YXJpYWJsZXMgaW4gc3RydWN0IHBhZ2UuCkFzIHdlbGwsIGlmIGRyaXZlciByZWxlYXNlcyB0aGUg cGFnZSBhZnRlciBpc29sYXRpb24gYnkgVk0sIHBhZ2UtPm1hcHBpbmcKZG9lc24ndCBoYXZlIHN0 YWJsZSB2YWx1ZSBhbHRob3VnaCBpdCBoYXMgUEFHRV9NQVBQSU5HX01PVkFCTEUKKExvb2sgYXQg X19DbGVhclBhZ2VNb3ZhYmxlKS4gQnV0IF9fUGFnZU1vdmFibGUgaXMgY2hlYXAgdG8gY2F0Y2gg d2hldGhlcgpwYWdlIGlzIExSVSBvciBub24tbHJ1IG1vdmFibGUgb25jZSB0aGUgcGFnZSBoYXMg YmVlbiBpc29sYXRlZC4gQmVjYXVzZQpMUlUgcGFnZXMgbmV2ZXIgY2FuIGhhdmUgUEFHRV9NQVBQ SU5HX01PVkFCTEUgaW4gcGFnZS0+bWFwcGluZy4gSXQgaXMgYWxzbwpnb29kIGZvciBqdXN0IHBl ZWtpbmcgdG8gdGVzdCBub24tbHJ1IG1vdmFibGUgcGFnZXMgYmVmb3JlIG1vcmUgZXhwZW5zaXZl CmNoZWNraW5nIHdpdGggbG9ja19wYWdlIGluIHBmbiBzY2FubmluZyB0byBzZWxlY3QgdmljdGlt LgoKRm9yIGd1YXJhbnRlZWluZyBub24tbHJ1IG1vdmFibGUgcGFnZSwgVk0gcHJvdmlkZXMgUGFn ZU1vdmFibGUgZnVuY3Rpb24uClVubGlrZSBfX1BhZ2VNb3ZhYmxlLCBQYWdlTW92YWJsZSBmdW5j dGlvbnMgdmFsaWRhdGVzIHBhZ2UtPm1hcHBpbmcgYW5kIAptYXBwaW5nLT5hX29wcy0+aXNvbGF0 ZV9wYWdlIHVuZGVyIGxvY2tfcGFnZS4gVGhlIGxvY2tfcGFnZSBwcmV2ZW50cyBzdWRkZW4KZGVz dHJveWluZyBvZiBwYWdlLT5tYXBwaW5nLgoKRHJpdmVyIHVzaW5nIF9fU2V0UGFnZU1vdmFibGUg c2hvdWxkIGNsZWFyIHRoZSBmbGFnIHZpYSBfX0NsZWFyTW92YWJsZVBhZ2UKdW5kZXIgcGFnZV9s b2NrIGJlZm9yZSB0aGUgcmVsZWFzaW5nIHRoZSBwYWdlLiIKCj4gCj4gPisJaWYgKHVubGlrZWx5 KCFpc19scnUpKSB7Cj4gPisJCVZNX0JVR19PTl9QQUdFKCFQYWdlSXNvbGF0ZWQocGFnZSksIHBh Z2UpOwo+ID4rCQlpZiAoIVBhZ2VNb3ZhYmxlKHBhZ2UpKSB7Cj4gPisJCQlyYyA9IE1JR1JBVEVQ QUdFX1NVQ0NFU1M7Cj4gPisJCQlfX0NsZWFyUGFnZUlzb2xhdGVkKHBhZ2UpOwo+ID4rCQkJZ290 byBvdXQ7Cj4gPisJCX0KPiA+Kwl9Cj4gPisKPiA+KwlpZiAobGlrZWx5KGlzX2xydSkpIHsKPiA+ KwkJaWYgKCFtYXBwaW5nKQo+ID4rCQkJcmMgPSBtaWdyYXRlX3BhZ2UobWFwcGluZywgbmV3cGFn ZSwgcGFnZSwgbW9kZSk7Cj4gPisJCWVsc2UgaWYgKG1hcHBpbmctPmFfb3BzLT5taWdyYXRlcGFn ZSkKPiA+KwkJCS8qCj4gPisJCQkgKiBNb3N0IHBhZ2VzIGhhdmUgYSBtYXBwaW5nIGFuZCBtb3N0 IGZpbGVzeXN0ZW1zCj4gPisJCQkgKiBwcm92aWRlIGEgbWlncmF0ZXBhZ2UgY2FsbGJhY2suIEFu b255bW91cyBwYWdlcwo+ID4rCQkJICogYXJlIHBhcnQgb2Ygc3dhcCBzcGFjZSB3aGljaCBhbHNv IGhhcyBpdHMgb3duCj4gPisJCQkgKiBtaWdyYXRlcGFnZSBjYWxsYmFjay4gVGhpcyBpcyB0aGUg bW9zdCBjb21tb24gcGF0aAo+ID4rCQkJICogZm9yIHBhZ2UgbWlncmF0aW9uLgo+ID4rCQkJICov Cj4gPisJCQlyYyA9IG1hcHBpbmctPmFfb3BzLT5taWdyYXRlcGFnZShtYXBwaW5nLCBuZXdwYWdl LAo+ID4rCQkJCQkJCXBhZ2UsIG1vZGUpOwo+ID4rCQllbHNlCj4gPisJCQlyYyA9IGZhbGxiYWNr X21pZ3JhdGVfcGFnZShtYXBwaW5nLCBuZXdwYWdlLAo+ID4rCQkJCQkJCXBhZ2UsIG1vZGUpOwo+ ID4rCX0gZWxzZSB7Cj4gPisJCXJjID0gbWFwcGluZy0+YV9vcHMtPm1pZ3JhdGVwYWdlKG1hcHBp bmcsIG5ld3BhZ2UsCj4gPisJCQkJCQlwYWdlLCBtb2RlKTsKPiA+KwkJV0FSTl9PTl9PTkNFKHJj ID09IE1JR1JBVEVQQUdFX1NVQ0NFU1MgJiYKPiA+KwkJCSFQYWdlSXNvbGF0ZWQocGFnZSkpOwo+ ID4rCX0KPiAKPiBXaHkgc3BsaXQgdGhlICFpc19scnUgaGFuZGxpbmcgaW4gdHdvIHBsYWNlcz8K CkZpeGVkLgoKPiAKPiA+Cj4gPiAJLyoKPiA+IAkgKiBXaGVuIHN1Y2Nlc3NmdWwsIG9sZCBwYWdl Y2FjaGUgcGFnZS0+bWFwcGluZyBtdXN0IGJlIGNsZWFyZWQgYmVmb3JlCj4gPiAJICogcGFnZSBp cyBmcmVlZDsgYnV0IHN0YXRzIHJlcXVpcmUgdGhhdCBQYWdlQW5vbiBiZSBsZWZ0IGFzIFBhZ2VB bm9uLgo+ID4gCSAqLwo+ID4gCWlmIChyYyA9PSBNSUdSQVRFUEFHRV9TVUNDRVNTKSB7Cj4gPi0J CWlmICghUGFnZUFub24ocGFnZSkpCj4gPisJCWlmIChfX1BhZ2VNb3ZhYmxlKHBhZ2UpKSB7Cj4g PisJCQlWTV9CVUdfT05fUEFHRSghUGFnZUlzb2xhdGVkKHBhZ2UpLCBwYWdlKTsKPiA+Kwo+ID4r CQkJLyoKPiA+KwkJCSAqIFdlIGNsZWFyIFBHX21vdmFibGUgdW5kZXIgcGFnZV9sb2NrIHNvIGFu eSBjb21wYWN0b3IKPiA+KwkJCSAqIGNhbm5vdCB0cnkgdG8gbWlncmF0ZSB0aGlzIHBhZ2UuCj4g PisJCQkgKi8KPiA+KwkJCV9fQ2xlYXJQYWdlSXNvbGF0ZWQocGFnZSk7Cj4gPisJCX0KPiA+Kwo+ ID4rCQlpZiAoISgodW5zaWduZWQgbG9uZylwYWdlLT5tYXBwaW5nICYgUEFHRV9NQVBQSU5HX0ZM QUdTKSkKPiA+IAkJCXBhZ2UtPm1hcHBpbmcgPSBOVUxMOwo+IAo+IFRoZSB0d28gbGluZXMgYWJv dmUgbWFrZSBsaXR0bGUgc2Vuc2UgdG8gbWUgd2l0aG91dCBhIGNvbW1lbnQuCgpJIGZvbGRlZCB0 aGlzLgoKQEAgLTkwMSw3ICs5MDEsMTIgQEAgc3RhdGljIGludCBtb3ZlX3RvX25ld19wYWdlKHN0 cnVjdCBwYWdlICpuZXdwYWdlLCBzdHJ1Y3QgcGFnZSAqcGFnZSwKICAgICAgICAgICAgICAgICAg ICAgICAgX19DbGVhclBhZ2VJc29sYXRlZChwYWdlKTsKICAgICAgICAgICAgICAgIH0KIAotICAg ICAgICAgICAgICAgaWYgKCEoKHVuc2lnbmVkIGxvbmcpcGFnZS0+bWFwcGluZyAmIFBBR0VfTUFQ UElOR19GTEFHUykpCisgICAgICAgICAgICAgICAvKgorICAgICAgICAgICAgICAgICogQW5vbnlt b3VzIGFuZCBtb3ZhYmxlIHBhZ2UtPm1hcHBpbmcgd2lsbCBiZSBjbGVhcmQgYnkKKyAgICAgICAg ICAgICAgICAqIGZyZWVfcGFnZXNfcHJlcGFyZSBzbyBkb24ndCByZXNldCBpdCBoZXJlIGZvciBr ZWVwaW5nCisgICAgICAgICAgICAgICAgKiB0aGUgdHlwZSB0byB3b3JrIFBhZ2VBbm9uLCBmb3Ig ZXhhbXBsZS4KKyAgICAgICAgICAgICAgICAqLworICAgICAgICAgICAgICAgaWYgKCFQYWdlTWFw cGluZ0ZsYWdzKHBhZ2UpKQogICAgICAgICAgICAgICAgICAgICAgICBwYWdlLT5tYXBwaW5nID0g TlVMTDsKICAgICAgICB9CiBvdXQ6CgoKPiBTaG91bGQgdGhlIGNvbmRpdGlvbiBiZSBuZWdhdGVk LCBldmVuPwoKTm8uCgo+IAo+IApfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwpkcmktZGV2ZWwgbWFpbGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVz a3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9k cmktZGV2ZWwK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f71.google.com (mail-pa0-f71.google.com [209.85.220.71]) by kanga.kvack.org (Postfix) with ESMTP id 3ECA66B0253 for ; Sun, 29 May 2016 21:33:01 -0400 (EDT) Received: by mail-pa0-f71.google.com with SMTP id di3so139937292pab.0 for ; Sun, 29 May 2016 18:33:01 -0700 (PDT) Received: from lgeamrelo13.lge.com (LGEAMRELO13.lge.com. [156.147.23.53]) by mx.google.com with ESMTP id xw5si5683484pac.189.2016.05.29.18.32.58 for ; Sun, 29 May 2016 18:32:59 -0700 (PDT) Date: Mon, 30 May 2016 10:33:27 +0900 From: Minchan Kim Subject: Re: [PATCH v6 02/12] mm: migrate: support non-lru movable page migration Message-ID: <20160530013327.GA8683@bbox> References: <1463754225-31311-1-git-send-email-minchan@kernel.org> <1463754225-31311-3-git-send-email-minchan@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Vlastimil Babka Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Rik van Riel , Joonsoo Kim , Mel Gorman , Hugh Dickins , Rafael Aquini , virtualization@lists.linux-foundation.org, Jonathan Corbet , John Einar Reitan , dri-devel@lists.freedesktop.org, Sergey Senozhatsky , Gioh Kim On Fri, May 27, 2016 at 04:26:21PM +0200, Vlastimil Babka wrote: > On 05/20/2016 04:23 PM, Minchan Kim wrote: > >We have allowed migration for only LRU pages until now and it was > >enough to make high-order pages. But recently, embedded system(e.g., > >webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) > >so we have seen several reports about troubles of small high-order > >allocation. For fixing the problem, there were several efforts > >(e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, > >reserved memory, vmalloc and so on) but if there are lots of > >non-movable pages in system, their solutions are void in the long run. > > > >So, this patch is to support facility to change non-movable pages > >with movable. For the feature, this patch introduces functions related > >to migration to address_space_operations as well as some page flags. > > > >If a driver want to make own pages movable, it should define three functions > >which are function pointers of struct address_space_operations. > > > >1. bool (*isolate_page) (struct page *page, isolate_mode_t mode); > > > >What VM expects on isolate_page function of driver is to return *true* > >if driver isolates page successfully. On returing true, VM marks the page > >as PG_isolated so concurrent isolation in several CPUs skip the page > >for isolation. If a driver cannot isolate the page, it should return *false*. > > > >Once page is successfully isolated, VM uses page.lru fields so driver > >shouldn't expect to preserve values in that fields. > > > >2. int (*migratepage) (struct address_space *mapping, > > struct page *newpage, struct page *oldpage, enum migrate_mode); > > > >After isolation, VM calls migratepage of driver with isolated page. > >The function of migratepage is to move content of the old page to new page > >and set up fields of struct page newpage. Keep in mind that you should > >clear PG_movable of oldpage via __ClearPageMovable under page_lock if you > >migrated the oldpage successfully and returns 0. > >If driver cannot migrate the page at the moment, driver can return -EAGAIN. > >On -EAGAIN, VM will retry page migration in a short time because VM interprets > >-EAGAIN as "temporal migration failure". On returning any error except -EAGAIN, > >VM will give up the page migration without retrying in this time. > > > >Driver shouldn't touch page.lru field VM using in the functions. > > > >3. void (*putback_page)(struct page *); > > > >If migration fails on isolated page, VM should return the isolated page > >to the driver so VM calls driver's putback_page with migration failed page. > >In this function, driver should put the isolated page back to the own data > >structure. > > > >4. non-lru movable page flags > > > >There are two page flags for supporting non-lru movable page. > > > >* PG_movable > > > >Driver should use the below function to make page movable under page_lock. > > > > void __SetPageMovable(struct page *page, struct address_space *mapping) > > > >It needs argument of address_space for registering migration family functions > >which will be called by VM. Exactly speaking, PG_movable is not a real flag of > >struct page. Rather than, VM reuses page->mapping's lower bits to represent it. > > > > #define PAGE_MAPPING_MOVABLE 0x2 > > page->mapping = page->mapping | PAGE_MAPPING_MOVABLE; > > Interesting, let's see how that works out... > > Overal this looks much better than the last version I checked! Thanks. > > [...] > > >@@ -357,29 +360,37 @@ PAGEFLAG(Idle, idle, PF_ANY) > > * with the PAGE_MAPPING_ANON bit set to distinguish it. See rmap.h. > > * > > * On an anonymous page in a VM_MERGEABLE area, if CONFIG_KSM is enabled, > >- * the PAGE_MAPPING_KSM bit may be set along with the PAGE_MAPPING_ANON bit; > >- * and then page->mapping points, not to an anon_vma, but to a private > >+ * the PAGE_MAPPING_MOVABLE bit may be set along with the PAGE_MAPPING_ANON > >+ * bit; and then page->mapping points, not to an anon_vma, but to a private > > * structure which KSM associates with that merged page. See ksm.h. > > * > >- * PAGE_MAPPING_KSM without PAGE_MAPPING_ANON is currently never used. > >+ * PAGE_MAPPING_KSM without PAGE_MAPPING_ANON is used for non-lru movable > >+ * page and then page->mapping points a struct address_space. > > * > > * Please note that, confusingly, "page_mapping" refers to the inode > > * address_space which maps the page from disk; whereas "page_mapped" > > * refers to user virtual address space into which the page is mapped. > > */ > >-#define PAGE_MAPPING_ANON 1 > >-#define PAGE_MAPPING_KSM 2 > >-#define PAGE_MAPPING_FLAGS (PAGE_MAPPING_ANON | PAGE_MAPPING_KSM) > >+#define PAGE_MAPPING_ANON 0x1 > >+#define PAGE_MAPPING_MOVABLE 0x2 > >+#define PAGE_MAPPING_KSM (PAGE_MAPPING_ANON | PAGE_MAPPING_MOVABLE) > >+#define PAGE_MAPPING_FLAGS (PAGE_MAPPING_ANON | PAGE_MAPPING_MOVABLE) > > > >-static __always_inline int PageAnonHead(struct page *page) > >+static __always_inline int PageMappingFlag(struct page *page) > > PageMappingFlags()? Yeb. > > [...] > > >diff --git a/mm/compaction.c b/mm/compaction.c > >index 1427366ad673..2d6862d0df60 100644 > >--- a/mm/compaction.c > >+++ b/mm/compaction.c > >@@ -81,6 +81,41 @@ static inline bool migrate_async_suitable(int migratetype) > > > > #ifdef CONFIG_COMPACTION > > > >+int PageMovable(struct page *page) > >+{ > >+ struct address_space *mapping; > >+ > >+ WARN_ON(!PageLocked(page)); > > Why not VM_BUG_ON_PAGE as elsewhere? I have backported this patchset to huge old kernel which doesn't have VM_BUG_ON_PAGE and forgot to correct it beforre sending mainline. Thanks. > > >+ if (!__PageMovable(page)) > >+ goto out; > > Just return 0. Maybe I love goto. You realized me. I will split up will her. > > >+ > >+ mapping = page_mapping(page); > >+ if (mapping && mapping->a_ops && mapping->a_ops->isolate_page) > >+ return 1; > >+out: > >+ return 0; > >+} > >+EXPORT_SYMBOL(PageMovable); > >+ > >+void __SetPageMovable(struct page *page, struct address_space *mapping) > >+{ > >+ VM_BUG_ON_PAGE(!PageLocked(page), page); > >+ VM_BUG_ON_PAGE((unsigned long)mapping & PAGE_MAPPING_MOVABLE, page); > >+ page->mapping = (void *)((unsigned long)mapping | PAGE_MAPPING_MOVABLE); > >+} > >+EXPORT_SYMBOL(__SetPageMovable); > >+ > >+void __ClearPageMovable(struct page *page) > >+{ > >+ VM_BUG_ON_PAGE(!PageLocked(page), page); > >+ VM_BUG_ON_PAGE(!PageMovable(page), page); > >+ VM_BUG_ON_PAGE(!((unsigned long)page->mapping & PAGE_MAPPING_MOVABLE), > >+ page); > > The last line sounds redundant, PageMovable() already checked this > via __PageMovable() Yeb. > > > >+ page->mapping = (void *)((unsigned long)page->mapping & > >+ PAGE_MAPPING_MOVABLE); > > This should be negated to clear... use ~PAGE_MAPPING_MOVABLE ? No. The intention is to clear only mapping value but PAGE_MAPPING_MOVABLE flag. So, any new migration trial will be failed because PageMovable checks page's mapping value but ongoing migraion handling can catch whether it's movable page or not with the type bit. For example, we need to keep the type bit to handle putback the page. With parallel freeing(e.g., __ClearPageMovable) from the owner after isolating the page, migration will be failed and put it back. Then, we need to identify movable to clear PG_isolate and shouldn't call mapping->a_ops->putback_page. For that, we need to keep the type bit until the page is return to buddy. > > >+} > >+EXPORT_SYMBOL(__ClearPageMovable); > >+ > > /* Do not skip compaction more than 64 times */ > > #define COMPACT_MAX_DEFER_SHIFT 6 > > > >@@ -735,21 +770,6 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, > > } > > > > /* > >- * Check may be lockless but that's ok as we recheck later. > >- * It's possible to migrate LRU pages and balloon pages > >- * Skip any other type of page > >- */ > >- is_lru = PageLRU(page); > >- if (!is_lru) { > >- if (unlikely(balloon_page_movable(page))) { > >- if (balloon_page_isolate(page)) { > >- /* Successfully isolated */ > >- goto isolate_success; > >- } > >- } > >- } > > So this effectively prevents movable compound pages from being > migrated. Are you sure no users of this functionality are going to > have compound pages? I assumed that they could, and so made the code > like this, with the is_lru variable (which is redundant after your > change). This implementation at the moment disables effectively non-lru compound page migration but I'm not a god so I can't make sure no one doesn't want it in future. If someone want it, we can support it then because this work doesn't prevent it by design. > > >- /* > > * Regardless of being on LRU, compound pages such as THP and > > * hugetlbfs are not to be compacted. We can potentially save > > * a lot of iterations if we skip them at once. The check is > >@@ -765,8 +785,38 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, > > goto isolate_fail; > > } > > > >- if (!is_lru) > >+ /* > >+ * Check may be lockless but that's ok as we recheck later. > >+ * It's possible to migrate LRU and non-lru movable pages. > >+ * Skip any other type of page > >+ */ > >+ is_lru = PageLRU(page); > >+ if (!is_lru) { > >+ if (unlikely(balloon_page_movable(page))) { > >+ if (balloon_page_isolate(page)) { > >+ /* Successfully isolated */ > >+ goto isolate_success; > >+ } > >+ } > > [...] > > >+bool isolate_movable_page(struct page *page, isolate_mode_t mode) > >+{ > >+ struct address_space *mapping; > >+ > >+ /* > >+ * Avoid burning cycles with pages that are yet under __free_pages(), > >+ * or just got freed under us. > >+ * > >+ * In case we 'win' a race for a movable page being freed under us and > >+ * raise its refcount preventing __free_pages() from doing its job > >+ * the put_page() at the end of this block will take care of > >+ * release this page, thus avoiding a nasty leakage. > >+ */ > >+ if (unlikely(!get_page_unless_zero(page))) > >+ goto out; > >+ > >+ /* > >+ * Check PageMovable before holding a PG_lock because page's owner > >+ * assumes anybody doesn't touch PG_lock of newly allocated page > >+ * so unconditionally grapping the lock ruins page's owner side. > >+ */ > >+ if (unlikely(!__PageMovable(page))) > >+ goto out_putpage; > >+ /* > >+ * As movable pages are not isolated from LRU lists, concurrent > >+ * compaction threads can race against page migration functions > >+ * as well as race against the releasing a page. > >+ * > >+ * In order to avoid having an already isolated movable page > >+ * being (wrongly) re-isolated while it is under migration, > >+ * or to avoid attempting to isolate pages being released, > >+ * lets be sure we have the page lock > >+ * before proceeding with the movable page isolation steps. > >+ */ > >+ if (unlikely(!trylock_page(page))) > >+ goto out_putpage; > >+ > >+ if (!PageMovable(page) || PageIsolated(page)) > >+ goto out_no_isolated; > >+ > >+ mapping = page_mapping(page); > > Hmm so on first tail page of a THP compound page, page->mapping will > alias with compound_mapcount. That can easily have a value matching > PageMovable flags and we'll proceed and start inspecting the > compound head in page_mapping()... maybe it's not a big deal, or we > better check and skip PageTail first, must think about it more... I thouht PageCompound check right before isolate_movable_page in isolate_migratepages_block will filter it out mostly but yeah it is racy without zone->lru_lock so it could reach to isolate_movable_page. However, PageMovable check in there investigates mapping, mapping->a_ops, and a_ops->isolate_page to verify whether it's movable page or not. I thought it's sufficient to filter THP page. > > [...] > > >@@ -755,33 +844,69 @@ static int move_to_new_page(struct page *newpage, struct page *page, > > enum migrate_mode mode) > > { > > struct address_space *mapping; > >- int rc; > >+ int rc = -EAGAIN; > >+ bool is_lru = !__PageMovable(page); > > > > VM_BUG_ON_PAGE(!PageLocked(page), page); > > VM_BUG_ON_PAGE(!PageLocked(newpage), newpage); > > > > mapping = page_mapping(page); > >- if (!mapping) > >- rc = migrate_page(mapping, newpage, page, mode); > >- else if (mapping->a_ops->migratepage) > >- /* > >- * Most pages have a mapping and most filesystems provide a > >- * migratepage callback. Anonymous pages are part of swap > >- * space which also has its own migratepage callback. This > >- * is the most common path for page migration. > >- */ > >- rc = mapping->a_ops->migratepage(mapping, newpage, page, mode); > >- else > >- rc = fallback_migrate_page(mapping, newpage, page, mode); > >+ /* > >+ * In case of non-lru page, it could be released after > >+ * isolation step. In that case, we shouldn't try > >+ * fallback migration which is designed for LRU pages. > >+ */ > > Hmm but is_lru was determined from !__PageMovable() above, also well > after the isolation step. So if the driver already released it, we > wouldn't detect it? And this function is all under same page lock, > so if __PageMovable was true above, so will be PageMovable below? You are missing what I mentioned above. We should keep the type bit to catch what you are saying(i.e., driver already released). __PageMovable just checks PAGE_MAPPING_MOVABLE flag and PageMovable checks page->mapping valid while __ClearPageMovable reset only valid vaule of mapping, not PAGE_MAPPING_MOVABLE flag. I wrote it down in Documentation/vm/page_migration. "For testing of non-lru movable page, VM supports __PageMovable function. However, it doesn't guarantee to identify non-lru movable page because page->mapping field is unified with other variables in struct page. As well, if driver releases the page after isolation by VM, page->mapping doesn't have stable value although it has PAGE_MAPPING_MOVABLE (Look at __ClearPageMovable). But __PageMovable is cheap to catch whether page is LRU or non-lru movable once the page has been isolated. Because LRU pages never can have PAGE_MAPPING_MOVABLE in page->mapping. It is also good for just peeking to test non-lru movable pages before more expensive checking with lock_page in pfn scanning to select victim. For guaranteeing non-lru movable page, VM provides PageMovable function. Unlike __PageMovable, PageMovable functions validates page->mapping and mapping->a_ops->isolate_page under lock_page. The lock_page prevents sudden destroying of page->mapping. Driver using __SetPageMovable should clear the flag via __ClearMovablePage under page_lock before the releasing the page." > > >+ if (unlikely(!is_lru)) { > >+ VM_BUG_ON_PAGE(!PageIsolated(page), page); > >+ if (!PageMovable(page)) { > >+ rc = MIGRATEPAGE_SUCCESS; > >+ __ClearPageIsolated(page); > >+ goto out; > >+ } > >+ } > >+ > >+ if (likely(is_lru)) { > >+ if (!mapping) > >+ rc = migrate_page(mapping, newpage, page, mode); > >+ else if (mapping->a_ops->migratepage) > >+ /* > >+ * Most pages have a mapping and most filesystems > >+ * provide a migratepage callback. Anonymous pages > >+ * are part of swap space which also has its own > >+ * migratepage callback. This is the most common path > >+ * for page migration. > >+ */ > >+ rc = mapping->a_ops->migratepage(mapping, newpage, > >+ page, mode); > >+ else > >+ rc = fallback_migrate_page(mapping, newpage, > >+ page, mode); > >+ } else { > >+ rc = mapping->a_ops->migratepage(mapping, newpage, > >+ page, mode); > >+ WARN_ON_ONCE(rc == MIGRATEPAGE_SUCCESS && > >+ !PageIsolated(page)); > >+ } > > Why split the !is_lru handling in two places? Fixed. > > > > > /* > > * When successful, old pagecache page->mapping must be cleared before > > * page is freed; but stats require that PageAnon be left as PageAnon. > > */ > > if (rc == MIGRATEPAGE_SUCCESS) { > >- if (!PageAnon(page)) > >+ if (__PageMovable(page)) { > >+ VM_BUG_ON_PAGE(!PageIsolated(page), page); > >+ > >+ /* > >+ * We clear PG_movable under page_lock so any compactor > >+ * cannot try to migrate this page. > >+ */ > >+ __ClearPageIsolated(page); > >+ } > >+ > >+ if (!((unsigned long)page->mapping & PAGE_MAPPING_FLAGS)) > > page->mapping = NULL; > > The two lines above make little sense to me without a comment. I folded this. @@ -901,7 +901,12 @@ static int move_to_new_page(struct page *newpage, struct page *page, __ClearPageIsolated(page); } - if (!((unsigned long)page->mapping & PAGE_MAPPING_FLAGS)) + /* + * Anonymous and movable page->mapping will be cleard by + * free_pages_prepare so don't reset it here for keeping + * the type to work PageAnon, for example. + */ + if (!PageMappingFlags(page)) page->mapping = NULL; } out: > Should the condition be negated, even? No. > > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753318AbcE3BdE (ORCPT ); Sun, 29 May 2016 21:33:04 -0400 Received: from LGEAMRELO13.lge.com ([156.147.23.53]:45085 "EHLO lgeamrelo13.lge.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751996AbcE3BdB (ORCPT ); Sun, 29 May 2016 21:33:01 -0400 X-Original-SENDERIP: 156.147.1.125 X-Original-MAILFROM: minchan@kernel.org X-Original-SENDERIP: 10.177.223.161 X-Original-MAILFROM: minchan@kernel.org Date: Mon, 30 May 2016 10:33:27 +0900 From: Minchan Kim To: Vlastimil Babka Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Rik van Riel , Joonsoo Kim , Mel Gorman , Hugh Dickins , Rafael Aquini , virtualization@lists.linux-foundation.org, Jonathan Corbet , John Einar Reitan , dri-devel@lists.freedesktop.org, Sergey Senozhatsky , Gioh Kim Subject: Re: [PATCH v6 02/12] mm: migrate: support non-lru movable page migration Message-ID: <20160530013327.GA8683@bbox> References: <1463754225-31311-1-git-send-email-minchan@kernel.org> <1463754225-31311-3-git-send-email-minchan@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, May 27, 2016 at 04:26:21PM +0200, Vlastimil Babka wrote: > On 05/20/2016 04:23 PM, Minchan Kim wrote: > >We have allowed migration for only LRU pages until now and it was > >enough to make high-order pages. But recently, embedded system(e.g., > >webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) > >so we have seen several reports about troubles of small high-order > >allocation. For fixing the problem, there were several efforts > >(e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, > >reserved memory, vmalloc and so on) but if there are lots of > >non-movable pages in system, their solutions are void in the long run. > > > >So, this patch is to support facility to change non-movable pages > >with movable. For the feature, this patch introduces functions related > >to migration to address_space_operations as well as some page flags. > > > >If a driver want to make own pages movable, it should define three functions > >which are function pointers of struct address_space_operations. > > > >1. bool (*isolate_page) (struct page *page, isolate_mode_t mode); > > > >What VM expects on isolate_page function of driver is to return *true* > >if driver isolates page successfully. On returing true, VM marks the page > >as PG_isolated so concurrent isolation in several CPUs skip the page > >for isolation. If a driver cannot isolate the page, it should return *false*. > > > >Once page is successfully isolated, VM uses page.lru fields so driver > >shouldn't expect to preserve values in that fields. > > > >2. int (*migratepage) (struct address_space *mapping, > > struct page *newpage, struct page *oldpage, enum migrate_mode); > > > >After isolation, VM calls migratepage of driver with isolated page. > >The function of migratepage is to move content of the old page to new page > >and set up fields of struct page newpage. Keep in mind that you should > >clear PG_movable of oldpage via __ClearPageMovable under page_lock if you > >migrated the oldpage successfully and returns 0. > >If driver cannot migrate the page at the moment, driver can return -EAGAIN. > >On -EAGAIN, VM will retry page migration in a short time because VM interprets > >-EAGAIN as "temporal migration failure". On returning any error except -EAGAIN, > >VM will give up the page migration without retrying in this time. > > > >Driver shouldn't touch page.lru field VM using in the functions. > > > >3. void (*putback_page)(struct page *); > > > >If migration fails on isolated page, VM should return the isolated page > >to the driver so VM calls driver's putback_page with migration failed page. > >In this function, driver should put the isolated page back to the own data > >structure. > > > >4. non-lru movable page flags > > > >There are two page flags for supporting non-lru movable page. > > > >* PG_movable > > > >Driver should use the below function to make page movable under page_lock. > > > > void __SetPageMovable(struct page *page, struct address_space *mapping) > > > >It needs argument of address_space for registering migration family functions > >which will be called by VM. Exactly speaking, PG_movable is not a real flag of > >struct page. Rather than, VM reuses page->mapping's lower bits to represent it. > > > > #define PAGE_MAPPING_MOVABLE 0x2 > > page->mapping = page->mapping | PAGE_MAPPING_MOVABLE; > > Interesting, let's see how that works out... > > Overal this looks much better than the last version I checked! Thanks. > > [...] > > >@@ -357,29 +360,37 @@ PAGEFLAG(Idle, idle, PF_ANY) > > * with the PAGE_MAPPING_ANON bit set to distinguish it. See rmap.h. > > * > > * On an anonymous page in a VM_MERGEABLE area, if CONFIG_KSM is enabled, > >- * the PAGE_MAPPING_KSM bit may be set along with the PAGE_MAPPING_ANON bit; > >- * and then page->mapping points, not to an anon_vma, but to a private > >+ * the PAGE_MAPPING_MOVABLE bit may be set along with the PAGE_MAPPING_ANON > >+ * bit; and then page->mapping points, not to an anon_vma, but to a private > > * structure which KSM associates with that merged page. See ksm.h. > > * > >- * PAGE_MAPPING_KSM without PAGE_MAPPING_ANON is currently never used. > >+ * PAGE_MAPPING_KSM without PAGE_MAPPING_ANON is used for non-lru movable > >+ * page and then page->mapping points a struct address_space. > > * > > * Please note that, confusingly, "page_mapping" refers to the inode > > * address_space which maps the page from disk; whereas "page_mapped" > > * refers to user virtual address space into which the page is mapped. > > */ > >-#define PAGE_MAPPING_ANON 1 > >-#define PAGE_MAPPING_KSM 2 > >-#define PAGE_MAPPING_FLAGS (PAGE_MAPPING_ANON | PAGE_MAPPING_KSM) > >+#define PAGE_MAPPING_ANON 0x1 > >+#define PAGE_MAPPING_MOVABLE 0x2 > >+#define PAGE_MAPPING_KSM (PAGE_MAPPING_ANON | PAGE_MAPPING_MOVABLE) > >+#define PAGE_MAPPING_FLAGS (PAGE_MAPPING_ANON | PAGE_MAPPING_MOVABLE) > > > >-static __always_inline int PageAnonHead(struct page *page) > >+static __always_inline int PageMappingFlag(struct page *page) > > PageMappingFlags()? Yeb. > > [...] > > >diff --git a/mm/compaction.c b/mm/compaction.c > >index 1427366ad673..2d6862d0df60 100644 > >--- a/mm/compaction.c > >+++ b/mm/compaction.c > >@@ -81,6 +81,41 @@ static inline bool migrate_async_suitable(int migratetype) > > > > #ifdef CONFIG_COMPACTION > > > >+int PageMovable(struct page *page) > >+{ > >+ struct address_space *mapping; > >+ > >+ WARN_ON(!PageLocked(page)); > > Why not VM_BUG_ON_PAGE as elsewhere? I have backported this patchset to huge old kernel which doesn't have VM_BUG_ON_PAGE and forgot to correct it beforre sending mainline. Thanks. > > >+ if (!__PageMovable(page)) > >+ goto out; > > Just return 0. Maybe I love goto. You realized me. I will split up will her. > > >+ > >+ mapping = page_mapping(page); > >+ if (mapping && mapping->a_ops && mapping->a_ops->isolate_page) > >+ return 1; > >+out: > >+ return 0; > >+} > >+EXPORT_SYMBOL(PageMovable); > >+ > >+void __SetPageMovable(struct page *page, struct address_space *mapping) > >+{ > >+ VM_BUG_ON_PAGE(!PageLocked(page), page); > >+ VM_BUG_ON_PAGE((unsigned long)mapping & PAGE_MAPPING_MOVABLE, page); > >+ page->mapping = (void *)((unsigned long)mapping | PAGE_MAPPING_MOVABLE); > >+} > >+EXPORT_SYMBOL(__SetPageMovable); > >+ > >+void __ClearPageMovable(struct page *page) > >+{ > >+ VM_BUG_ON_PAGE(!PageLocked(page), page); > >+ VM_BUG_ON_PAGE(!PageMovable(page), page); > >+ VM_BUG_ON_PAGE(!((unsigned long)page->mapping & PAGE_MAPPING_MOVABLE), > >+ page); > > The last line sounds redundant, PageMovable() already checked this > via __PageMovable() Yeb. > > > >+ page->mapping = (void *)((unsigned long)page->mapping & > >+ PAGE_MAPPING_MOVABLE); > > This should be negated to clear... use ~PAGE_MAPPING_MOVABLE ? No. The intention is to clear only mapping value but PAGE_MAPPING_MOVABLE flag. So, any new migration trial will be failed because PageMovable checks page's mapping value but ongoing migraion handling can catch whether it's movable page or not with the type bit. For example, we need to keep the type bit to handle putback the page. With parallel freeing(e.g., __ClearPageMovable) from the owner after isolating the page, migration will be failed and put it back. Then, we need to identify movable to clear PG_isolate and shouldn't call mapping->a_ops->putback_page. For that, we need to keep the type bit until the page is return to buddy. > > >+} > >+EXPORT_SYMBOL(__ClearPageMovable); > >+ > > /* Do not skip compaction more than 64 times */ > > #define COMPACT_MAX_DEFER_SHIFT 6 > > > >@@ -735,21 +770,6 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, > > } > > > > /* > >- * Check may be lockless but that's ok as we recheck later. > >- * It's possible to migrate LRU pages and balloon pages > >- * Skip any other type of page > >- */ > >- is_lru = PageLRU(page); > >- if (!is_lru) { > >- if (unlikely(balloon_page_movable(page))) { > >- if (balloon_page_isolate(page)) { > >- /* Successfully isolated */ > >- goto isolate_success; > >- } > >- } > >- } > > So this effectively prevents movable compound pages from being > migrated. Are you sure no users of this functionality are going to > have compound pages? I assumed that they could, and so made the code > like this, with the is_lru variable (which is redundant after your > change). This implementation at the moment disables effectively non-lru compound page migration but I'm not a god so I can't make sure no one doesn't want it in future. If someone want it, we can support it then because this work doesn't prevent it by design. > > >- /* > > * Regardless of being on LRU, compound pages such as THP and > > * hugetlbfs are not to be compacted. We can potentially save > > * a lot of iterations if we skip them at once. The check is > >@@ -765,8 +785,38 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, > > goto isolate_fail; > > } > > > >- if (!is_lru) > >+ /* > >+ * Check may be lockless but that's ok as we recheck later. > >+ * It's possible to migrate LRU and non-lru movable pages. > >+ * Skip any other type of page > >+ */ > >+ is_lru = PageLRU(page); > >+ if (!is_lru) { > >+ if (unlikely(balloon_page_movable(page))) { > >+ if (balloon_page_isolate(page)) { > >+ /* Successfully isolated */ > >+ goto isolate_success; > >+ } > >+ } > > [...] > > >+bool isolate_movable_page(struct page *page, isolate_mode_t mode) > >+{ > >+ struct address_space *mapping; > >+ > >+ /* > >+ * Avoid burning cycles with pages that are yet under __free_pages(), > >+ * or just got freed under us. > >+ * > >+ * In case we 'win' a race for a movable page being freed under us and > >+ * raise its refcount preventing __free_pages() from doing its job > >+ * the put_page() at the end of this block will take care of > >+ * release this page, thus avoiding a nasty leakage. > >+ */ > >+ if (unlikely(!get_page_unless_zero(page))) > >+ goto out; > >+ > >+ /* > >+ * Check PageMovable before holding a PG_lock because page's owner > >+ * assumes anybody doesn't touch PG_lock of newly allocated page > >+ * so unconditionally grapping the lock ruins page's owner side. > >+ */ > >+ if (unlikely(!__PageMovable(page))) > >+ goto out_putpage; > >+ /* > >+ * As movable pages are not isolated from LRU lists, concurrent > >+ * compaction threads can race against page migration functions > >+ * as well as race against the releasing a page. > >+ * > >+ * In order to avoid having an already isolated movable page > >+ * being (wrongly) re-isolated while it is under migration, > >+ * or to avoid attempting to isolate pages being released, > >+ * lets be sure we have the page lock > >+ * before proceeding with the movable page isolation steps. > >+ */ > >+ if (unlikely(!trylock_page(page))) > >+ goto out_putpage; > >+ > >+ if (!PageMovable(page) || PageIsolated(page)) > >+ goto out_no_isolated; > >+ > >+ mapping = page_mapping(page); > > Hmm so on first tail page of a THP compound page, page->mapping will > alias with compound_mapcount. That can easily have a value matching > PageMovable flags and we'll proceed and start inspecting the > compound head in page_mapping()... maybe it's not a big deal, or we > better check and skip PageTail first, must think about it more... I thouht PageCompound check right before isolate_movable_page in isolate_migratepages_block will filter it out mostly but yeah it is racy without zone->lru_lock so it could reach to isolate_movable_page. However, PageMovable check in there investigates mapping, mapping->a_ops, and a_ops->isolate_page to verify whether it's movable page or not. I thought it's sufficient to filter THP page. > > [...] > > >@@ -755,33 +844,69 @@ static int move_to_new_page(struct page *newpage, struct page *page, > > enum migrate_mode mode) > > { > > struct address_space *mapping; > >- int rc; > >+ int rc = -EAGAIN; > >+ bool is_lru = !__PageMovable(page); > > > > VM_BUG_ON_PAGE(!PageLocked(page), page); > > VM_BUG_ON_PAGE(!PageLocked(newpage), newpage); > > > > mapping = page_mapping(page); > >- if (!mapping) > >- rc = migrate_page(mapping, newpage, page, mode); > >- else if (mapping->a_ops->migratepage) > >- /* > >- * Most pages have a mapping and most filesystems provide a > >- * migratepage callback. Anonymous pages are part of swap > >- * space which also has its own migratepage callback. This > >- * is the most common path for page migration. > >- */ > >- rc = mapping->a_ops->migratepage(mapping, newpage, page, mode); > >- else > >- rc = fallback_migrate_page(mapping, newpage, page, mode); > >+ /* > >+ * In case of non-lru page, it could be released after > >+ * isolation step. In that case, we shouldn't try > >+ * fallback migration which is designed for LRU pages. > >+ */ > > Hmm but is_lru was determined from !__PageMovable() above, also well > after the isolation step. So if the driver already released it, we > wouldn't detect it? And this function is all under same page lock, > so if __PageMovable was true above, so will be PageMovable below? You are missing what I mentioned above. We should keep the type bit to catch what you are saying(i.e., driver already released). __PageMovable just checks PAGE_MAPPING_MOVABLE flag and PageMovable checks page->mapping valid while __ClearPageMovable reset only valid vaule of mapping, not PAGE_MAPPING_MOVABLE flag. I wrote it down in Documentation/vm/page_migration. "For testing of non-lru movable page, VM supports __PageMovable function. However, it doesn't guarantee to identify non-lru movable page because page->mapping field is unified with other variables in struct page. As well, if driver releases the page after isolation by VM, page->mapping doesn't have stable value although it has PAGE_MAPPING_MOVABLE (Look at __ClearPageMovable). But __PageMovable is cheap to catch whether page is LRU or non-lru movable once the page has been isolated. Because LRU pages never can have PAGE_MAPPING_MOVABLE in page->mapping. It is also good for just peeking to test non-lru movable pages before more expensive checking with lock_page in pfn scanning to select victim. For guaranteeing non-lru movable page, VM provides PageMovable function. Unlike __PageMovable, PageMovable functions validates page->mapping and mapping->a_ops->isolate_page under lock_page. The lock_page prevents sudden destroying of page->mapping. Driver using __SetPageMovable should clear the flag via __ClearMovablePage under page_lock before the releasing the page." > > >+ if (unlikely(!is_lru)) { > >+ VM_BUG_ON_PAGE(!PageIsolated(page), page); > >+ if (!PageMovable(page)) { > >+ rc = MIGRATEPAGE_SUCCESS; > >+ __ClearPageIsolated(page); > >+ goto out; > >+ } > >+ } > >+ > >+ if (likely(is_lru)) { > >+ if (!mapping) > >+ rc = migrate_page(mapping, newpage, page, mode); > >+ else if (mapping->a_ops->migratepage) > >+ /* > >+ * Most pages have a mapping and most filesystems > >+ * provide a migratepage callback. Anonymous pages > >+ * are part of swap space which also has its own > >+ * migratepage callback. This is the most common path > >+ * for page migration. > >+ */ > >+ rc = mapping->a_ops->migratepage(mapping, newpage, > >+ page, mode); > >+ else > >+ rc = fallback_migrate_page(mapping, newpage, > >+ page, mode); > >+ } else { > >+ rc = mapping->a_ops->migratepage(mapping, newpage, > >+ page, mode); > >+ WARN_ON_ONCE(rc == MIGRATEPAGE_SUCCESS && > >+ !PageIsolated(page)); > >+ } > > Why split the !is_lru handling in two places? Fixed. > > > > > /* > > * When successful, old pagecache page->mapping must be cleared before > > * page is freed; but stats require that PageAnon be left as PageAnon. > > */ > > if (rc == MIGRATEPAGE_SUCCESS) { > >- if (!PageAnon(page)) > >+ if (__PageMovable(page)) { > >+ VM_BUG_ON_PAGE(!PageIsolated(page), page); > >+ > >+ /* > >+ * We clear PG_movable under page_lock so any compactor > >+ * cannot try to migrate this page. > >+ */ > >+ __ClearPageIsolated(page); > >+ } > >+ > >+ if (!((unsigned long)page->mapping & PAGE_MAPPING_FLAGS)) > > page->mapping = NULL; > > The two lines above make little sense to me without a comment. I folded this. @@ -901,7 +901,12 @@ static int move_to_new_page(struct page *newpage, struct page *page, __ClearPageIsolated(page); } - if (!((unsigned long)page->mapping & PAGE_MAPPING_FLAGS)) + /* + * Anonymous and movable page->mapping will be cleard by + * free_pages_prepare so don't reset it here for keeping + * the type to work PageAnon, for example. + */ + if (!PageMappingFlags(page)) page->mapping = NULL; } out: > Should the condition be negated, even? No. > >