From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Tue, 3 Jan 2017 09:27:25 -0700 From: Ross Zwisler Subject: Re: [PATCH] x86: fix kaslr and memmap collision Message-ID: <20170103162725.GB13904@linux.intel.com> References: <147977413859.13657.2181994710415174471.stgit@djiang5-desk3.ch.intel.com> <20161122084754.GA25596@gmail.com> <20170103083137.GA15788@x1> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20170103083137.GA15788@x1> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Baoquan He Cc: Kees Cook , "linux-nvdimm@lists.01.org" , X86 ML , david , "linux-kernel@vger.kernel.org" , Ingo Molnar , Ingo Molnar , "H. Peter Anvin" , dyoung@redhat.com, Thomas Gleixner List-ID: T24gVHVlLCBKYW4gMDMsIDIwMTcgYXQgMDQ6MzE6MzdQTSArMDgwMCwgQmFvcXVhbiBIZSB3cm90 ZToKPiBIaSBEYW4sCj4gCj4gT24gMTEvMjIvMTYgYXQgMDk6MjZhbSwgRGFuIFdpbGxpYW1zIHdy b3RlOgo+ID4gWyByZXBseWluZyBmb3IgRGF2ZSBzaW5jZSBoZSdzIG9mZmxpbmUgdG9kYXkgYW5k IHRvbW9ycm93IF0KPiA+IAo+ID4gT24gVHVlLCBOb3YgMjIsIDIwMTYgYXQgMTI6NDcgQU0sIElu Z28gTW9sbmFyIDxtaW5nb0BrZXJuZWwub3JnPiB3cm90ZToKPiA+ID4KPiA+ID4gKiBEYXZlIEpp YW5nIDxkYXZlLmppYW5nQGludGVsLmNvbT4gd3JvdGU6Cj4gPiA+Cj4gPiA+PiBDT05GSUdfUkFO RE9NSVpFX0JBU0UgcmVsb2NhdGVzIHRoZSBrZXJuZWwgdG8gYSByYW5kb20gYmFzZSBhZGRyZXNz Lgo+ID4gPj4gSG93ZXZlciBpdCBkb2VzIG5vdCB0YWtlIGludG8gYWNjb3VudCB0aGUgbWVtbWFw PSBwYXJhbWV0ZXIgcGFzc2VkIGluIGZyb20KPiA+ID4+IHRoZSBrZXJuZWwgY29tbWFuZGxpbmUu Cj4gPiA+Cj4gPiA+IG1lbW1hcD0gcGFyYW1ldGVycyBhcmUgb2Z0ZW4gdXNlZCBhcyBhIGxpc3Qu Cj4gPiA+Cj4gPiA+PiBbLi4uXSBUaGlzIHJlc3VsdHMgaW4gdGhlIGtlcm5lbCBzb21ldGltZXMg YmVpbmcgcHV0IGluIHRoZSBtaWRkbGUgb2YgdGhlIHVzZXIKPiA+ID4+IG1lbW1hcC4gWy4uLl0K PiA+ID4KPiA+ID4gV2hhdCBkb2VzIHRoaXMgbWVhbj8gSWYgbWVtbWFwPSBpcyB1c2VkIHRvIHJl LWRlZmluZSB0aGUgbWVtb3J5IG1hcCB0aGVuIHRoZQo+ID4gPiBrZXJuZWwgZ2V0dGluZyBpbiB0 aGUgbWlkZGxlIG9mIGEgUkFNIGFyZWEgaXMgd2hhdCB3ZSB3YW50LCBpc24ndCBpdD8gV2hhdCB3 ZQo+ID4gPiBkb24ndCB3YW50IGlzIGZvciB0aGUga2VybmVsIHRvIGdldCBpbnRvIHJlc2VydmVk IGFyZWFzLCByaWdodD8KPiA+IAo+ID4gUmlnaHQsIHRoaXMgaXMgYWJvdXQgdGVhY2hpbmcga2Fz bHIgdG8gbm90IGxhbmQgdGhlIGtlcm5lbCBpbiBuZXdseQo+ID4gZGVmaW5lZCByZXNlcnZlZCBy ZWdpb25zIHRoYXQgd2VyZSBub3QgbWFya2VkIHJlc2VydmVkIGluIHRoZSBpbml0aWFsCj4gPiBl ODIwIG1hcCBmcm9tIHBsYXRmb3JtIGZpcm13YXJlLgo+IAo+IElmIG9ubHkgdGVsbCBrYXNsciB0 byBub3QgbGFuZCBrZXJuZWwgaW4gbmV3bHkgZGVmaW5lZCByZXNlcnZlZCByZWdpb25zLAo+IG1l bW9yeSBhZGRlZCBieSAibWVtbWFwPW5uW0tNR11Ac3NbS01HXSIgc2hvdWxkIG5vdCBiZSBhdm9p ZGVkIHNpbmNlCj4gaXQncyB1c2FibGUgbWVtb3J5LiBLZXJuZWwgcmFuZG9taXplZCBpbnRvIHRo aXMgcmVnaW9uIGlzIGFsc28gd2hhdCB3ZQo+IHdhbnQuIE5vdCBzdXJlIGlmIEkgdW5kZXJzdGFu ZCBpdCByaWdodC4KClRoZSBmb2xsb3dpbmcgdGV4dCBpcyBmcm9tOgpodHRwczovL252ZGltbS53 aWtpLmtlcm5lbC5vcmcvaG93X3RvX2Nob29zZV90aGVfY29ycmVjdF9tZW1tYXBfa2VybmVsX3Bh cmFtZXRlcl9mb3JfcG1lbV9vbl95b3VyX3N5c3RlbQoKSG9wZWZ1bGx5IHRoaXMgd2lsbCBtYWtl IGl0IGNsZWFyZXIuCgotLS0KCkFub3RoZXIgdGhpbmcgdGhhdCB5b3UgbWF5IG5lZWQgdG8gYmUg YXdhcmUgb2YgaXMgdGhlIENPTkZJR19SQU5ET01JWkVfQkFTRQprZXJuZWwgY29uZmlnIG9wdGlv bi4gV2hlbiBlbmFibGVkLCB0aGlzIHJhbmRvbWl6ZXMgdGhlIHBoeXNpY2FsIGFkZHJlc3MgYXQK d2hpY2ggdGhlIGtlcm5lbCBpbWFnZSBpcyBkZWNvbXByZXNzZWQgYW5kIHRoZSB2aXJ0dWFsIGFk ZHJlc3Mgd2hlcmUgdGhlIGtlcm5lbAppbWFnZSBpcyBtYXBwZWQuIEN1cnJlbnRseSB0aGlzIHJh bmRvbSBhZGRyZXNzIGlzIGNob3NlbiB3aXRob3V0IHJlZ2FyZCB0byB0aGUKbWVtbWFwIGtlcm5l bCBjb21tYW5kIGxpbmUgcGFyYW1ldGVyLgoKVGhpcyBtZWFucyB0aGF0IHRoZSBrZXJuZWwgY2Fu IGNob29zZSB0byBwdXQgaXRzZWxmIGluIHRoZSBtaWRkbGUgb2YgeW91cgpyZXNlcnZlZCBtZW1t YXAgYXJlYS4gWW91IGNhbiBvYnNlcnZlIHRoaXMgYmVoYXZpb3IgdmlhIC9wcm9jL2lvbWVtLgoK SGVyZSBpcyAvcHJvYy9pb21lbSBmcm9tIGEgc3lzdGVtIHdpdGggQ09ORklHX1JBTkRPTUlaRV9C QVNFIHR1cm5lZCBvZmY6CgojIGNhdCAvcHJvYy9pb21lbQowMDAwMDAwMC0wMDAwMGZmZiA6IHJl c2VydmVkCjAwMDAxMDAwLTAwMDlmYmZmIDogU3lzdGVtIFJBTQowMDA5ZmMwMC0wMDA5ZmZmZiA6 IHJlc2VydmVkCjAwMGEwMDAwLTAwMGJmZmZmIDogUENJIEJ1cyAwMDAwOjAwCjAwMGMwMDAwLTAw MGM5N2ZmIDogVmlkZW8gUk9NCjAwMGM5ODAwLTAwMGNhNWZmIDogQWRhcHRlciBST00KMDAwY2E4 MDAtMDAwY2NiZmYgOiBBZGFwdGVyIFJPTQowMDBmMDAwMC0wMDBmZmZmZiA6IHJlc2VydmVkCiAg MDAwZjAwMDAtMDAwZmZmZmYgOiBTeXN0ZW0gUk9NCjAwMTAwMDAwLWJmZmQ4ZmZmIDogU3lzdGVt IFJBTQogIDAxMDAwMDAwLTAxYjE4NTk4IDogS2VybmVsIGNvZGUKICAwMWIxODU5OS0wMjNmNTNm ZiA6IEtlcm5lbCBkYXRhCiAgMDI3NmQwMDAtMDM2NWVmZmYgOiBLZXJuZWwgYnNzCmJmZmQ5MDAw LWJmZmZmZmZmIDogcmVzZXJ2ZWQKYzAwMDAwMDAtZmViZmZmZmYgOiBQQ0kgQnVzIDAwMDA6MDAK ICBmNDAwMDAwMC1mN2ZmZmZmZiA6IDAwMDA6MDA6MDIuMAogIGY4MDAwMDAwLWZiZmZmZmZmIDog MDAwMDowMDowMi4wCiAgZmMwMDAwMDAtZmMwM2ZmZmYgOiAwMDAwOjAwOjAzLjAKICBmYzA1MDAw MC1mYzA1MWZmZiA6IDAwMDA6MDA6MDIuMAogIGZjMDUyMDAwLWZjMDUyZmZmIDogMDAwMDowMDow My4wCiAgZmMwNTMwMDAtZmMwNTNmZmYgOiAwMDAwOjAwOjA0LjAKICBmYzA1NDAwMC1mYzA1NGZm ZiA6IDAwMDA6MDA6MDUuNwogICAgZmMwNTQwMDAtZmMwNTRmZmYgOiBlaGNpX2hjZAogIGZjMDU1 MDAwLWZjMDU1ZmZmIDogMDAwMDowMDowNi4wCmZlYzAwMDAwLWZlYzAwM2ZmIDogSU9BUElDIDAK ZmVlMDAwMDAtZmVlMDBmZmYgOiBMb2NhbCBBUElDCmZlZmZjMDAwLWZlZmZmZmZmIDogcmVzZXJ2 ZWQKZmZmYzAwMDAtZmZmZmZmZmYgOiByZXNlcnZlZAoxMDAwMDAwMDAtNGZmZmZmZmZmIDogUGVy c2lzdGVudCBNZW1vcnkgKGxlZ2FjeSkKICAxMDAwMDAwMDAtNGZmZmZmZmZmIDogbmFtZXNwYWNl MC4wCjUwMDAwMDAwMC01M2ZmZmZmZmYgOiBTeXN0ZW0gUkFNCgpUaGUgaW50ZXJlc3RpbmcgYml0 cyBmb3IgdXMgYXJlIHRoZSDigJxTeXN0ZW0gUkFN4oCdIHJlZ2lvbiBmcm9tIDAwMTAwMDAwLWJm ZmQ4ZmZmLAphbmQgdGhlIOKAnFBlcnNpc3RlbnQgTWVtb3J5IChsZWdhY3kp4oCdIHJlZ2lvbiBm cm9tIDEwMDAwMDAwMC00ZmZmZmZmZmYuCgpJZiBJIHR1cm4gb24gQ09ORklHX1JBTkRPTUlaRV9C QVNFIG9uIHRoaXMgc2FtZSBzeXN0ZW0sIEkgZ2V0IHRoZSBmb2xsb3dpbmc6CgojIGNhdCAvcHJv Yy9pb21lbQowMDAwMDAwMC0wMDAwMGZmZiA6IHJlc2VydmVkCjAwMDAxMDAwLTAwMDlmYmZmIDog U3lzdGVtIFJBTQowMDA5ZmMwMC0wMDA5ZmZmZiA6IHJlc2VydmVkCjAwMGEwMDAwLTAwMGJmZmZm IDogUENJIEJ1cyAwMDAwOjAwCjAwMGMwMDAwLTAwMGM5N2ZmIDogVmlkZW8gUk9NCjAwMGM5ODAw LTAwMGNhNWZmIDogQWRhcHRlciBST00KMDAwY2E4MDAtMDAwY2NiZmYgOiBBZGFwdGVyIFJPTQow MDBmMDAwMC0wMDBmZmZmZiA6IHJlc2VydmVkCiAgMDAwZjAwMDAtMDAwZmZmZmYgOiBTeXN0ZW0g Uk9NCjAwMTAwMDAwLWJmZmQ4ZmZmIDogU3lzdGVtIFJBTQpiZmZkOTAwMC1iZmZmZmZmZiA6IHJl c2VydmVkCmMwMDAwMDAwLWZlYmZmZmZmIDogUENJIEJ1cyAwMDAwOjAwCiAgZjQwMDAwMDAtZjdm ZmZmZmYgOiAwMDAwOjAwOjAyLjAKICBmODAwMDAwMC1mYmZmZmZmZiA6IDAwMDA6MDA6MDIuMAog IGZjMDAwMDAwLWZjMDNmZmZmIDogMDAwMDowMDowMy4wCiAgZmMwNTAwMDAtZmMwNTFmZmYgOiAw MDAwOjAwOjAyLjAKICBmYzA1MjAwMC1mYzA1MmZmZiA6IDAwMDA6MDA6MDMuMAogIGZjMDUzMDAw LWZjMDUzZmZmIDogMDAwMDowMDowNC4wCiAgZmMwNTQwMDAtZmMwNTRmZmYgOiAwMDAwOjAwOjA1 LjcKICAgIGZjMDU0MDAwLWZjMDU0ZmZmIDogZWhjaV9oY2QKICBmYzA1NTAwMC1mYzA1NWZmZiA6 IDAwMDA6MDA6MDYuMApmZWMwMDAwMC1mZWMwMDNmZiA6IElPQVBJQyAwCmZlZTAwMDAwLWZlZTAw ZmZmIDogTG9jYWwgQVBJQwpmZWZmYzAwMC1mZWZmZmZmZiA6IHJlc2VydmVkCmZmZmMwMDAwLWZm ZmZmZmZmIDogcmVzZXJ2ZWQKMTAwMDAwMDAwLTRlNmZmZmZmZiA6IFBlcnNpc3RlbnQgTWVtb3J5 IChsZWdhY3kpCjRlNzAwMDAwMC00ZTk2OGJmZmYgOiBTeXN0ZW0gUkFNCiAgNGU3MDAwMDAwLTRl N2IxODVkOCA6IEtlcm5lbCBjb2RlCiAgNGU3YjE4NWQ5LTRlODNmNTRiZiA6IEtlcm5lbCBkYXRh CiAgNGU4NzZkMDAwLTRlOTY1ZWZmZiA6IEtlcm5lbCBic3MKNGU5NjhjMDAwLTRmZmZmZmZmZiA6 IFBlcnNpc3RlbnQgTWVtb3J5IChsZWdhY3kpCjUwMDAwMDAwMC01M2ZmZmZmZmYgOiBTeXN0ZW0g UkFNCgpUaGUg4oCcU3lzdGVtIFJBTeKAnSByZWdpb24gbm93IHNpdHMgaW4gdGhlIG1pZGRsZSBv ZiBteSDigJxQZXJzaXN0ZW50IE1lbW9yeQoobGVnYWN5KeKAnSByZWdpb24sIHNwbGl0dGluZyBp dCBpbiBoYWxmLiBUaGlzIHJlc3VsdHMgaW4gdGhlIGZvbGxvd2luZyBrZXJuZWwKV0FSTklORzoK ClsgICAgNi4zNTYxODBdIFdBUk5JTkc6IENQVTogNCBQSUQ6IDY4OSBhdCBrZXJuZWwvbWVtcmVt YXAuYzozMDAgZGV2bV9tZW1yZW1hcF9wYWdlcysweDNiMi8weDRjMApbICAgIDYuMzU3NzU3XSBk ZXZtX21lbXJlbWFwX3BhZ2VzIGF0dGVtcHRlZCBvbiBtaXhlZCByZWdpb24gW21lbSAweDRlOTY4 YzAwMC0weDRmZmZmZmZmZiBmbGFncyAweDIwMF0KCmFuZCBubyAvZGV2L3BtZW0qIGRldmljZXMg YmVpbmcgY3JlYXRlZC4KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KTGludXgtbnZkaW1tIG1haWxpbmcgbGlzdApMaW51eC1udmRpbW1AbGlzdHMuMDEub3Jn Cmh0dHBzOi8vbGlzdHMuMDEub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtbnZkaW1tCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965002AbdACQ2B (ORCPT ); Tue, 3 Jan 2017 11:28:01 -0500 Received: from mga06.intel.com ([134.134.136.31]:57017 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759080AbdACQ1o (ORCPT ); Tue, 3 Jan 2017 11:27:44 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,455,1477983600"; d="scan'208";a="49055051" Date: Tue, 3 Jan 2017 09:27:25 -0700 From: Ross Zwisler To: Baoquan He Cc: Dan Williams , Kees Cook , "linux-nvdimm@lists.01.org" , X86 ML , david , "linux-kernel@vger.kernel.org" , Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , dyoung@redhat.com, Ingo Molnar Subject: Re: [PATCH] x86: fix kaslr and memmap collision Message-ID: <20170103162725.GB13904@linux.intel.com> References: <147977413859.13657.2181994710415174471.stgit@djiang5-desk3.ch.intel.com> <20161122084754.GA25596@gmail.com> <20170103083137.GA15788@x1> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20170103083137.GA15788@x1> User-Agent: Mutt/1.7.1 (2016-10-04) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jan 03, 2017 at 04:31:37PM +0800, Baoquan He wrote: > Hi Dan, > > On 11/22/16 at 09:26am, Dan Williams wrote: > > [ replying for Dave since he's offline today and tomorrow ] > > > > On Tue, Nov 22, 2016 at 12:47 AM, Ingo Molnar wrote: > > > > > > * Dave Jiang wrote: > > > > > >> CONFIG_RANDOMIZE_BASE relocates the kernel to a random base address. > > >> However it does not take into account the memmap= parameter passed in from > > >> the kernel commandline. > > > > > > memmap= parameters are often used as a list. > > > > > >> [...] This results in the kernel sometimes being put in the middle of the user > > >> memmap. [...] > > > > > > What does this mean? If memmap= is used to re-define the memory map then the > > > kernel getting in the middle of a RAM area is what we want, isn't it? What we > > > don't want is for the kernel to get into reserved areas, right? > > > > Right, this is about teaching kaslr to not land the kernel in newly > > defined reserved regions that were not marked reserved in the initial > > e820 map from platform firmware. > > If only tell kaslr to not land kernel in newly defined reserved regions, > memory added by "memmap=nn[KMG]@ss[KMG]" should not be avoided since > it's usable memory. Kernel randomized into this region is also what we > want. Not sure if I understand it right. The following text is from: https://nvdimm.wiki.kernel.org/how_to_choose_the_correct_memmap_kernel_parameter_for_pmem_on_your_system Hopefully this will make it clearer. --- Another thing that you may need to be aware of is the CONFIG_RANDOMIZE_BASE kernel config option. When enabled, this randomizes the physical address at which the kernel image is decompressed and the virtual address where the kernel image is mapped. Currently this random address is chosen without regard to the memmap kernel command line parameter. This means that the kernel can choose to put itself in the middle of your reserved memmap area. You can observe this behavior via /proc/iomem. Here is /proc/iomem from a system with CONFIG_RANDOMIZE_BASE turned off: # cat /proc/iomem 00000000-00000fff : reserved 00001000-0009fbff : System RAM 0009fc00-0009ffff : reserved 000a0000-000bffff : PCI Bus 0000:00 000c0000-000c97ff : Video ROM 000c9800-000ca5ff : Adapter ROM 000ca800-000ccbff : Adapter ROM 000f0000-000fffff : reserved 000f0000-000fffff : System ROM 00100000-bffd8fff : System RAM 01000000-01b18598 : Kernel code 01b18599-023f53ff : Kernel data 0276d000-0365efff : Kernel bss bffd9000-bfffffff : reserved c0000000-febfffff : PCI Bus 0000:00 f4000000-f7ffffff : 0000:00:02.0 f8000000-fbffffff : 0000:00:02.0 fc000000-fc03ffff : 0000:00:03.0 fc050000-fc051fff : 0000:00:02.0 fc052000-fc052fff : 0000:00:03.0 fc053000-fc053fff : 0000:00:04.0 fc054000-fc054fff : 0000:00:05.7 fc054000-fc054fff : ehci_hcd fc055000-fc055fff : 0000:00:06.0 fec00000-fec003ff : IOAPIC 0 fee00000-fee00fff : Local APIC feffc000-feffffff : reserved fffc0000-ffffffff : reserved 100000000-4ffffffff : Persistent Memory (legacy) 100000000-4ffffffff : namespace0.0 500000000-53fffffff : System RAM The interesting bits for us are the “System RAM” region from 00100000-bffd8fff, and the “Persistent Memory (legacy)” region from 100000000-4ffffffff. If I turn on CONFIG_RANDOMIZE_BASE on this same system, I get the following: # cat /proc/iomem 00000000-00000fff : reserved 00001000-0009fbff : System RAM 0009fc00-0009ffff : reserved 000a0000-000bffff : PCI Bus 0000:00 000c0000-000c97ff : Video ROM 000c9800-000ca5ff : Adapter ROM 000ca800-000ccbff : Adapter ROM 000f0000-000fffff : reserved 000f0000-000fffff : System ROM 00100000-bffd8fff : System RAM bffd9000-bfffffff : reserved c0000000-febfffff : PCI Bus 0000:00 f4000000-f7ffffff : 0000:00:02.0 f8000000-fbffffff : 0000:00:02.0 fc000000-fc03ffff : 0000:00:03.0 fc050000-fc051fff : 0000:00:02.0 fc052000-fc052fff : 0000:00:03.0 fc053000-fc053fff : 0000:00:04.0 fc054000-fc054fff : 0000:00:05.7 fc054000-fc054fff : ehci_hcd fc055000-fc055fff : 0000:00:06.0 fec00000-fec003ff : IOAPIC 0 fee00000-fee00fff : Local APIC feffc000-feffffff : reserved fffc0000-ffffffff : reserved 100000000-4e6ffffff : Persistent Memory (legacy) 4e7000000-4e968bfff : System RAM 4e7000000-4e7b185d8 : Kernel code 4e7b185d9-4e83f54bf : Kernel data 4e876d000-4e965efff : Kernel bss 4e968c000-4ffffffff : Persistent Memory (legacy) 500000000-53fffffff : System RAM The “System RAM” region now sits in the middle of my “Persistent Memory (legacy)” region, splitting it in half. This results in the following kernel WARNING: [ 6.356180] WARNING: CPU: 4 PID: 689 at kernel/memremap.c:300 devm_memremap_pages+0x3b2/0x4c0 [ 6.357757] devm_memremap_pages attempted on mixed region [mem 0x4e968c000-0x4ffffffff flags 0x200] and no /dev/pmem* devices being created.