From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f71.google.com (mail-oi0-f71.google.com [209.85.218.71]) by kanga.kvack.org (Postfix) with ESMTP id 20D7C6B0261 for ; Fri, 3 Nov 2017 13:05:24 -0400 (EDT) Received: by mail-oi0-f71.google.com with SMTP id j126so3387294oib.9 for ; Fri, 03 Nov 2017 10:05:24 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id u63si3251833oib.316.2017.11.03.10.05.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Nov 2017 10:05:23 -0700 (PDT) From: Florian Weimer Subject: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: Date: Fri, 3 Nov 2017 18:05:20 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: linuxppc-dev@lists.ozlabs.org, linux-mm We are seeing an issue on ppc64le and ppc64 (and perhaps on some arm variant, but I have not seen it on our own builders) where running localedef as part of the glibc build crashes with a segmentation fault. Kernel version is 4.13.9 (Fedora 26 variant). I have only seen this with an explicit loader invocation, like this: while I18NPATH=. /lib64/ld64.so.1 /usr/bin/localedef --alias-file=../intl/locale.alias --no-archive -i locales/nl_AW -c -f charmaps/UTF-8 --prefix=/builddir/build/BUILDROOT/glibc-2.26-16.fc27.ppc64 nl_AW ; do : ; done To be run in the localedata subdirectory of a glibc *source* tree, after a build. You may have to create the /builddir/build/BUILDROOT/glibc-2.26-16.fc27.ppc64/usr/lib/locale directory. I have only reproduced this inside a Fedora 27 chroot on a Fedora 26 host, but there it does not matter if you run the old (chroot) or newly built binary. I filed this as a glibc bug for tracking: https://sourceware.org/bugzilla/show_bug.cgi?id=22390 There's an strace log and a coredump from the crash. I think the data shows that the address in question should be writable. The crossed 0x0000800000000000 binary is very suggestive. I think that based on the operation of glibc's malloc, this write would be the first time this happens during the lifetime of the process. Does that ring any bells? Is there anything I can do to provide more data? The host is an LPAR with a stock Fedora 26 kernel, so I can use any diagnostics tool which is provided by Fedora. I can try to come up with a better reproducer, but that appears to be difficult. Thanks, Florian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id 9A2A06B0253 for ; Sun, 5 Nov 2017 07:19:09 -0500 (EST) Received: by mail-pf0-f199.google.com with SMTP id b79so7969103pfk.9 for ; Sun, 05 Nov 2017 04:19:09 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j11sor3011952plt.137.2017.11.05.04.19.07 for (Google Transport Security); Sun, 05 Nov 2017 04:19:07 -0800 (PST) Date: Sun, 5 Nov 2017 23:18:50 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171105231850.5e313e46@roar.ozlabs.ibm.com> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer , "Aneesh Kumar K . V" Cc: linuxppc-dev@lists.ozlabs.org, linux-mm On Fri, 3 Nov 2017 18:05:20 +0100 Florian Weimer wrote: > We are seeing an issue on ppc64le and ppc64 (and perhaps on some arm > variant, but I have not seen it on our own builders) where running > localedef as part of the glibc build crashes with a segmentation fault. > > Kernel version is 4.13.9 (Fedora 26 variant). > > I have only seen this with an explicit loader invocation, like this: > > while I18NPATH=. /lib64/ld64.so.1 /usr/bin/localedef > --alias-file=../intl/locale.alias --no-archive -i locales/nl_AW -c -f > charmaps/UTF-8 > --prefix=/builddir/build/BUILDROOT/glibc-2.26-16.fc27.ppc64 nl_AW ; do : > ; done > > To be run in the localedata subdirectory of a glibc *source* tree, after > a build. You may have to create the > /builddir/build/BUILDROOT/glibc-2.26-16.fc27.ppc64/usr/lib/locale > directory. I have only reproduced this inside a Fedora 27 chroot on a > Fedora 26 host, but there it does not matter if you run the old (chroot) > or newly built binary. > > I filed this as a glibc bug for tracking: > > https://sourceware.org/bugzilla/show_bug.cgi?id=22390 > > There's an strace log and a coredump from the crash. > > I think the data shows that the address in question should be writable. > > The crossed 0x0000800000000000 binary is very suggestive. I think that > based on the operation of glibc's malloc, this write would be the first > time this happens during the lifetime of the process. > > Does that ring any bells? Is there anything I can do to provide more > data? The host is an LPAR with a stock Fedora 26 kernel, so I can use > any diagnostics tool which is provided by Fedora. There was a recent change to move to 128TB address space by default, and option for 512TB addresses if explicitly requested. Your brk request asked for > 128TB which the kernel gave it, but the address limit in the paca that the SLB miss tests against was not updated to reflect the switch to 512TB address space. Why is your brk starting so high? Are you trying to test the > 128TB case, or maybe something is confused by the 64->128TB change? What's the strace look like if you run on a distro or <= 4.10 kernel? Something like the following patch may help if you could test. Thanks, Nick --- arch/powerpc/mm/hugetlbpage-radix.c | 18 ++++++++++++++---- arch/powerpc/mm/mmap.c | 34 +++++++++++++++++++++++++--------- arch/powerpc/mm/mmu_context_book3s64.c | 14 +++++++------- arch/powerpc/mm/slice.c | 28 +++++++++++++++++++--------- 4 files changed, 65 insertions(+), 29 deletions(-) diff --git a/arch/powerpc/mm/hugetlbpage-radix.c b/arch/powerpc/mm/hugetlbpage-radix.c index a12e86395025..44e1109765b5 100644 --- a/arch/powerpc/mm/hugetlbpage-radix.c +++ b/arch/powerpc/mm/hugetlbpage-radix.c @@ -50,8 +50,16 @@ radix__hugetlb_get_unmapped_area(struct file *file, unsigned long addr, struct hstate *h = hstate_file(file); struct vm_unmapped_area_info info; - if (unlikely(addr > mm->context.addr_limit && addr < TASK_SIZE)) - mm->context.addr_limit = TASK_SIZE; + /* + * If address is specified explicitly and crosses addr_limit, or if + * address is unspecified but len is greater than addr_limit, then + * expand out to TASK_SIZE. + */ + if (unlikely(addr + len >= mm->context.addr_limit)) { + if ((!addr || addr + len > mm->context.addr_limit) && + mm->context.addr_limit != TASK_SIZE) + mm->context.addr_limit = TASK_SIZE; + } if (len & ~huge_page_mask(h)) return -EINVAL; @@ -82,8 +90,10 @@ radix__hugetlb_get_unmapped_area(struct file *file, unsigned long addr, info.align_mask = PAGE_MASK & ~huge_page_mask(h); info.align_offset = 0; - if (addr > DEFAULT_MAP_WINDOW) - info.high_limit += mm->context.addr_limit - DEFAULT_MAP_WINDOW; + if (addr + len >= DEFAULT_MAP_WINDOW) { + if (!addr || addr + len > DEFAULT_MAP_WINDOW) + info.high_limit += mm->context.addr_limit - DEFAULT_MAP_WINDOW; + } return vm_unmapped_area(&info); } diff --git a/arch/powerpc/mm/mmap.c b/arch/powerpc/mm/mmap.c index 5d78b193fec4..a8fe1eaf1d96 100644 --- a/arch/powerpc/mm/mmap.c +++ b/arch/powerpc/mm/mmap.c @@ -108,9 +108,16 @@ radix__arch_get_unmapped_area(struct file *filp, unsigned long addr, struct vm_area_struct *vma; struct vm_unmapped_area_info info; - if (unlikely(addr > mm->context.addr_limit && - mm->context.addr_limit != TASK_SIZE)) - mm->context.addr_limit = TASK_SIZE; + /* + * If address is specified explicitly and crosses addr_limit, or if + * address is unspecified but len is greater than addr_limit, then + * expand out to TASK_SIZE. + */ + if (unlikely(addr + len >= mm->context.addr_limit)) { + if ((!addr || addr + len > mm->context.addr_limit) && + mm->context.addr_limit != TASK_SIZE) + mm->context.addr_limit = TASK_SIZE; + } if (len > mm->task_size - mmap_min_addr) return -ENOMEM; @@ -131,7 +138,7 @@ radix__arch_get_unmapped_area(struct file *filp, unsigned long addr, info.low_limit = mm->mmap_base; info.align_mask = 0; - if (unlikely(addr > DEFAULT_MAP_WINDOW)) + if (unlikely(addr + len > DEFAULT_MAP_WINDOW)) info.high_limit = mm->context.addr_limit; else info.high_limit = DEFAULT_MAP_WINDOW; @@ -151,9 +158,16 @@ radix__arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr = addr0; struct vm_unmapped_area_info info; - if (unlikely(addr > mm->context.addr_limit && - mm->context.addr_limit != TASK_SIZE)) - mm->context.addr_limit = TASK_SIZE; + /* + * If address is specified explicitly and crosses addr_limit, or if + * address is unspecified but len is greater than addr_limit, then + * expand out to TASK_SIZE. + */ + if (unlikely(addr + len >= mm->context.addr_limit)) { + if ((!addr || addr + len > mm->context.addr_limit) && + mm->context.addr_limit != TASK_SIZE) + mm->context.addr_limit = TASK_SIZE; + } /* requested length too big for entire address space */ if (len > mm->task_size - mmap_min_addr) @@ -177,8 +191,10 @@ radix__arch_get_unmapped_area_topdown(struct file *filp, info.high_limit = mm->mmap_base; info.align_mask = 0; - if (addr > DEFAULT_MAP_WINDOW) - info.high_limit += mm->context.addr_limit - DEFAULT_MAP_WINDOW; + if (addr + len >= DEFAULT_MAP_WINDOW) { + if (!addr || addr + len > DEFAULT_MAP_WINDOW) + info.high_limit += mm->context.addr_limit - DEFAULT_MAP_WINDOW; + } addr = vm_unmapped_area(&info); if (!(addr & ~PAGE_MASK)) diff --git a/arch/powerpc/mm/mmu_context_book3s64.c b/arch/powerpc/mm/mmu_context_book3s64.c index 05e15386d4cb..1116ea0ddb2e 100644 --- a/arch/powerpc/mm/mmu_context_book3s64.c +++ b/arch/powerpc/mm/mmu_context_book3s64.c @@ -92,13 +92,6 @@ static int hash__init_new_context(struct mm_struct *mm) if (index < 0) return index; - /* - * We do switch_slb() early in fork, even before we setup the - * mm->context.addr_limit. Default to max task size so that we copy the - * default values to paca which will help us to handle slb miss early. - */ - mm->context.addr_limit = DEFAULT_MAP_WINDOW_USER64; - /* * The old code would re-promote on fork, we don't do that when using * slices as it could cause problem promoting slices that have been @@ -162,6 +155,13 @@ int init_new_context(struct task_struct *tsk, struct mm_struct *mm) if (index < 0) return index; + /* + * In the case of exec, use the default limit, + * otherwise inherit it from the mm we are duplicating. + */ + if (!mm->context.addr_limit) + mm->context.addr_limit = DEFAULT_MAP_WINDOW_USER64; + mm->context.id = index; #ifdef CONFIG_PPC_64K_PAGES diff --git a/arch/powerpc/mm/slice.c b/arch/powerpc/mm/slice.c index 45f6740dd407..aa55523b6759 100644 --- a/arch/powerpc/mm/slice.c +++ b/arch/powerpc/mm/slice.c @@ -329,7 +329,7 @@ static unsigned long slice_find_area_topdown(struct mm_struct *mm, * Only for that request for which high_limit is above * DEFAULT_MAP_WINDOW we should apply this. */ - if (high_limit > DEFAULT_MAP_WINDOW) + if (high_limit > DEFAULT_MAP_WINDOW) addr += mm->context.addr_limit - DEFAULT_MAP_WINDOW; while (addr > PAGE_SIZE) { @@ -418,19 +418,29 @@ unsigned long slice_get_unmapped_area(unsigned long addr, unsigned long len, /* * Check if we need to expland slice area. + * + * If address is specified explicitly and crosses addr_limit, or if + * address is unspecified but len is greater than addr_limit, then + * expand out to TASK_SIZE. */ - if (unlikely(addr > mm->context.addr_limit && - mm->context.addr_limit != TASK_SIZE)) { - mm->context.addr_limit = TASK_SIZE; - on_each_cpu(slice_flush_segments, mm, 1); + if (unlikely(addr + len >= mm->context.addr_limit)) { + if ((!addr || addr + len > mm->context.addr_limit) && + mm->context.addr_limit != TASK_SIZE) { + mm->context.addr_limit = TASK_SIZE; + on_each_cpu(slice_flush_segments, mm, 1); + } } + /* * This mmap request can allocate upt to 512TB */ - if (addr > DEFAULT_MAP_WINDOW) - high_limit = mm->context.addr_limit; - else - high_limit = DEFAULT_MAP_WINDOW; + high_limit = DEFAULT_MAP_WINDOW; + if (addr + len >= DEFAULT_MAP_WINDOW) { + if (!addr || addr + len > DEFAULT_MAP_WINDOW) + high_limit = mm->context.addr_limit; + } + + /* * init different masks */ -- 2.15.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f72.google.com (mail-oi0-f72.google.com [209.85.218.72]) by kanga.kvack.org (Postfix) with ESMTP id F31866B0253 for ; Sun, 5 Nov 2017 07:35:44 -0500 (EST) Received: by mail-oi0-f72.google.com with SMTP id e123so7642117oig.14 for ; Sun, 05 Nov 2017 04:35:44 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id v19si4735902oia.5.2017.11.05.04.35.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 05 Nov 2017 04:35:44 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> From: Florian Weimer Message-ID: <919a1cb5-c3b5-ddee-d6a6-0994c282ae84@redhat.com> Date: Sun, 5 Nov 2017 13:35:40 +0100 MIME-Version: 1.0 In-Reply-To: <20171105231850.5e313e46@roar.ozlabs.ibm.com> Content-Type: multipart/mixed; boundary="------------B8C2898AF8C15A8B4627E032" Content-Language: en-US Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin , "Aneesh Kumar K . V" Cc: linuxppc-dev@lists.ozlabs.org, linux-mm This is a multi-part message in MIME format. --------------B8C2898AF8C15A8B4627E032 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 11/05/2017 01:18 PM, Nicholas Piggin wrote: > There was a recent change to move to 128TB address space by default, > and option for 512TB addresses if explicitly requested. Do you have a commit hash for the introduction of 128TB by default? Thanks. > Your brk request asked for > 128TB which the kernel gave it, but the > address limit in the paca that the SLB miss tests against was not > updated to reflect the switch to 512TB address space. > > Why is your brk starting so high? Are you trying to test the > 128TB > case, or maybe something is confused by the 64->128TB change? What's > the strace look like if you run on a distro or <= 4.10 kernel? I think it is a consequence of running with an explicit loader invocation. With that, the heap is placed above ld.so, which can be quite high in the address space. I'm attaching two runs of cat, one executing directly as /bin/cat, and one with /lib64/ld64.so.1 /bin/cat. Fortunately, this does *not* apply to PIE binaries (also attached). However, explicit loader invocations are sometimes used in test suites (not just for glibc), and these sporadic test failures are quite annoying. Do you still need the strace log? And if yes, of what exactly? > Something like the following patch may help if you could test. Okay, this will take some time. Thanks, Florian --------------B8C2898AF8C15A8B4627E032 Content-Type: text/plain; charset=UTF-8; name="pie.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="pie.txt" MTIzMWQwMDAwLTEyMzFlMDAwMCByLXhwIDAwMDAwMDAwIGZkOjAwIDE3ODUyNDI1ICAgICAg ICAgICAgICAgICAgICAgICAgIC9yb290L2Eub3V0CjEyMzFlMDAwMC0xMjMxZjAwMDAgci0t cCAwMDAwMDAwMCBmZDowMCAxNzg1MjQyNSAgICAgICAgICAgICAgICAgICAgICAgICAvcm9v dC9hLm91dAoxMjMxZjAwMDAtMTIzMjAwMDAwIHJ3LXAgMDAwMTAwMDAgZmQ6MDAgMTc4NTI0 MjUgICAgICAgICAgICAgICAgICAgICAgICAgL3Jvb3QvYS5vdXQKMTAwMGRiYzAwMDAtMTAw MGRiZjAwMDAgcnctcCAwMDAwMDAwMCAwMDowMCAwICAgICAgICAgICAgICAgICAgICAgICAg ICAgIFtoZWFwXQo3ZmZmYTMxZDAwMDAtN2ZmZmEzNDAwMDAwIHIteHAgMDAwMDAwMDAgZmQ6 MDAgMjUxNjc5MzYgICAgICAgICAgICAgICAgICAgL3Vzci9saWI2NC9wb3dlcjgvbGliYy0y LjI1LnNvCjdmZmZhMzQwMDAwMC03ZmZmYTM0MTAwMDAgLS0tcCAwMDIzMDAwMCBmZDowMCAy NTE2NzkzNiAgICAgICAgICAgICAgICAgICAvdXNyL2xpYjY0L3Bvd2VyOC9saWJjLTIuMjUu c28KN2ZmZmEzNDEwMDAwLTdmZmZhMzQyMDAwMCByLS1wIDAwMjMwMDAwIGZkOjAwIDI1MTY3 OTM2ICAgICAgICAgICAgICAgICAgIC91c3IvbGliNjQvcG93ZXI4L2xpYmMtMi4yNS5zbwo3 ZmZmYTM0MjAwMDAtN2ZmZmEzNDMwMDAwIHJ3LXAgMDAyNDAwMDAgZmQ6MDAgMjUxNjc5MzYg ICAgICAgICAgICAgICAgICAgL3Vzci9saWI2NC9wb3dlcjgvbGliYy0yLjI1LnNvCjdmZmZh MzQ0MDAwMC03ZmZmYTM0NjAwMDAgci14cCAwMDAwMDAwMCAwMDowMCAwICAgICAgICAgICAg ICAgICAgICAgICAgICBbdmRzb10KN2ZmZmEzNDYwMDAwLTdmZmZhMzRhMDAwMCByLXhwIDAw MDAwMDAwIGZkOjAwIDgzOTAzMjkgICAgICAgICAgICAgICAgICAgIC91c3IvbGliNjQvbGQt Mi4yNS5zbwo3ZmZmYTM0YTAwMDAtN2ZmZmEzNGIwMDAwIHItLXAgMDAwMzAwMDAgZmQ6MDAg ODM5MDMyOSAgICAgICAgICAgICAgICAgICAgL3Vzci9saWI2NC9sZC0yLjI1LnNvCjdmZmZh MzRiMDAwMC03ZmZmYTM0YzAwMDAgcnctcCAwMDA0MDAwMCBmZDowMCA4MzkwMzI5ICAgICAg ICAgICAgICAgICAgICAvdXNyL2xpYjY0L2xkLTIuMjUuc28KN2ZmZmU5NDUwMDAwLTdmZmZl OTQ4MDAwMCBydy1wIDAwMDAwMDAwIDAwOjAwIDAgICAgICAgICAgICAgICAgICAgICAgICAg IFtzdGFja10K --------------B8C2898AF8C15A8B4627E032 Content-Type: text/plain; charset=UTF-8; name="explicit-ldso.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="explicit-ldso.txt" N2ZmZjdlNzkwMDAwLTdmZmY3ZTdkMDAwMCBydy1wIDAwMDAwMDAwIDAwOjAwIDAgCjdmZmY3 ZTdkMDAwMC03ZmZmN2U4MzAwMDAgci0tcCAwMDAwMDAwMCBmZDowMCAyNTE2NzkyNSAgICAg ICAgICAgICAgICAgICAvdXNyL2xpYi9sb2NhbGUvZW5fVVMudXRmOC9MQ19DVFlQRQo3ZmZm N2U4MzAwMDAtN2ZmZjdlODQwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMjUxNjc5MjggICAg ICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfTlVNRVJJQwo3 ZmZmN2U4NDAwMDAtN2ZmZjdlODUwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5Mjkg ICAgICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfVElNRQo3 ZmZmN2U4NTAwMDAtN2ZmZjdlOTgwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMjUxNjc5MjQg ICAgICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfQ09MTEFU RQo3ZmZmN2U5ODAwMDAtN2ZmZjdlOTkwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5 MjcgICAgICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfTU9O RVRBUlkKN2ZmZjdlOTkwMDAwLTdmZmY3ZTlhMDAwMCByLS1wIDAwMDAwMDAwIGZkOjAwIDI1 MTEgICAgICAgICAgICAgICAgICAgICAgIC91c3IvbGliL2xvY2FsZS9lbl9VUy51dGY4L0xD X01FU1NBR0VTL1NZU19MQ19NRVNTQUdFUwo3ZmZmN2U5YTAwMDAtN2ZmZjdlOWIwMDAwIHIt LXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5NDIgICAgICAgICAgICAgICAgICAgL3Vzci9saWIv bG9jYWxlL2VuX1VTLnV0ZjgvTENfUEFQRVIKN2ZmZjdlOWIwMDAwLTdmZmY3ZTljMDAwMCBy LS1wIDAwMDAwMDAwIGZkOjAwIDI1MTY3OTI3ICAgICAgICAgICAgICAgICAgIC91c3IvbGli L2xvY2FsZS9lbl9VUy51dGY4L0xDX05BTUUKN2ZmZjdlOWMwMDAwLTdmZmY3ZTlkMDAwMCBy LS1wIDAwMDAwMDAwIGZkOjAwIDE2Nzk4OTI0ICAgICAgICAgICAgICAgICAgIC91c3IvbGli L2xvY2FsZS9lbl9VUy51dGY4L0xDX0FERFJFU1MKN2ZmZjdlOWQwMDAwLTdmZmY3ZTllMDAw MCByLS1wIDAwMDAwMDAwIGZkOjAwIDE2Nzk4OTI4ICAgICAgICAgICAgICAgICAgIC91c3Iv bGliL2xvY2FsZS9lbl9VUy51dGY4L0xDX1RFTEVQSE9ORQo3ZmZmN2U5ZTAwMDAtN2ZmZjdl OWYwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5MjYgICAgICAgICAgICAgICAgICAg L3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfTUVBU1VSRU1FTlQKN2ZmZjdlOWYwMDAw LTdmZmY3ZWEwMDAwMCByLS1zIDAwMDAwMDAwIGZkOjAwIDgzOTA2NjkgICAgICAgICAgICAg ICAgICAgIC91c3IvbGliNjQvZ2NvbnYvZ2NvbnYtbW9kdWxlcy5jYWNoZQo3ZmZmN2VhMDAw MDAtN2ZmZjdlYzMwMDAwIHIteHAgMDAwMDAwMDAgZmQ6MDAgMjUxNjc5MzYgICAgICAgICAg ICAgICAgICAgL3Vzci9saWI2NC9wb3dlcjgvbGliYy0yLjI1LnNvCjdmZmY3ZWMzMDAwMC03 ZmZmN2VjNDAwMDAgLS0tcCAwMDIzMDAwMCBmZDowMCAyNTE2NzkzNiAgICAgICAgICAgICAg ICAgICAvdXNyL2xpYjY0L3Bvd2VyOC9saWJjLTIuMjUuc28KN2ZmZjdlYzQwMDAwLTdmZmY3 ZWM1MDAwMCByLS1wIDAwMjMwMDAwIGZkOjAwIDI1MTY3OTM2ICAgICAgICAgICAgICAgICAg IC91c3IvbGliNjQvcG93ZXI4L2xpYmMtMi4yNS5zbwo3ZmZmN2VjNTAwMDAtN2ZmZjdlYzYw MDAwIHJ3LXAgMDAyNDAwMDAgZmQ6MDAgMjUxNjc5MzYgICAgICAgICAgICAgICAgICAgL3Vz ci9saWI2NC9wb3dlcjgvbGliYy0yLjI1LnNvCjdmZmY3ZWM2MDAwMC03ZmZmN2VjNzAwMDAg ci0tcCAwMDAwMDAwMCBmZDowMCAxNjc5ODkyNSAgICAgICAgICAgICAgICAgICAvdXNyL2xp Yi9sb2NhbGUvZW5fVVMudXRmOC9MQ19JREVOVElGSUNBVElPTgo3ZmZmN2VjNzAwMDAtN2Zm ZjdlYzgwMDAwIHIteHAgMDAwMDAwMDAgZmQ6MDAgMjAyMjkzICAgICAgICAgICAgICAgICAg ICAgL3Vzci9iaW4vY2F0CjdmZmY3ZWM4MDAwMC03ZmZmN2VjOTAwMDAgci0tcCAwMDAwMDAw MCBmZDowMCAyMDIyOTMgICAgICAgICAgICAgICAgICAgICAvdXNyL2Jpbi9jYXQKN2ZmZjdl YzkwMDAwLTdmZmY3ZWNhMDAwMCBydy1wIDAwMDEwMDAwIGZkOjAwIDIwMjI5MyAgICAgICAg ICAgICAgICAgICAgIC91c3IvYmluL2NhdAo3ZmZmN2VjYTAwMDAtN2ZmZjdlY2MwMDAwIHIt eHAgMDAwMDAwMDAgMDA6MDAgMCAgICAgICAgICAgICAgICAgICAgICAgICAgW3Zkc29dCjdm ZmY3ZWNjMDAwMC03ZmZmN2VkMDAwMDAgci14cCAwMDAwMDAwMCBmZDowMCA4MzkwMzI5ICAg ICAgICAgICAgICAgICAgICAvdXNyL2xpYjY0L2xkLTIuMjUuc28KN2ZmZjdlZDAwMDAwLTdm ZmY3ZWQxMDAwMCByLS1wIDAwMDMwMDAwIGZkOjAwIDgzOTAzMjkgICAgICAgICAgICAgICAg ICAgIC91c3IvbGliNjQvbGQtMi4yNS5zbwo3ZmZmN2VkMTAwMDAtN2ZmZjdlZDIwMDAwIHJ3 LXAgMDAwNDAwMDAgZmQ6MDAgODM5MDMyOSAgICAgICAgICAgICAgICAgICAgL3Vzci9saWI2 NC9sZC0yLjI1LnNvCjdmZmY5YmUyMDAwMC03ZmZmOWJlNTAwMDAgcnctcCAwMDAwMDAwMCAw MDowMCAwICAgICAgICAgICAgICAgICAgICAgICAgICBbaGVhcF0KN2ZmZmZkNDcwMDAwLTdm ZmZmZDRhMDAwMCBydy1wIDAwMDAwMDAwIDAwOjAwIDAgICAgICAgICAgICAgICAgICAgICAg ICAgIFtzdGFja10K --------------B8C2898AF8C15A8B4627E032 Content-Type: text/plain; charset=UTF-8; name="direct.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="direct.txt" MTNlZWEwMDAwLTEzZWViMDAwMCByLXhwIDAwMDAwMDAwIGZkOjAwIDIwMjI5MyAgICAgICAg ICAgICAgICAgICAgICAgICAgIC91c3IvYmluL2NhdAoxM2VlYjAwMDAtMTNlZWMwMDAwIHIt LXAgMDAwMDAwMDAgZmQ6MDAgMjAyMjkzICAgICAgICAgICAgICAgICAgICAgICAgICAgL3Vz ci9iaW4vY2F0CjEzZWVjMDAwMC0xM2VlZDAwMDAgcnctcCAwMDAxMDAwMCBmZDowMCAyMDIy OTMgICAgICAgICAgICAgICAgICAgICAgICAgICAvdXNyL2Jpbi9jYXQKMTAwM2ViZTAwMDAt MTAwM2VjMTAwMDAgcnctcCAwMDAwMDAwMCAwMDowMCAwICAgICAgICAgICAgICAgICAgICAg ICAgICAgIFtoZWFwXQo3ZmZmOTc0OTAwMDAtN2ZmZjk3NGQwMDAwIHJ3LXAgMDAwMDAwMDAg MDA6MDAgMCAKN2ZmZjk3NGQwMDAwLTdmZmY5NzUzMDAwMCByLS1wIDAwMDAwMDAwIGZkOjAw IDI1MTY3OTI1ICAgICAgICAgICAgICAgICAgIC91c3IvbGliL2xvY2FsZS9lbl9VUy51dGY4 L0xDX0NUWVBFCjdmZmY5NzUzMDAwMC03ZmZmOTc1NDAwMDAgci0tcCAwMDAwMDAwMCBmZDow MCAyNTE2NzkyOCAgICAgICAgICAgICAgICAgICAvdXNyL2xpYi9sb2NhbGUvZW5fVVMudXRm OC9MQ19OVU1FUklDCjdmZmY5NzU0MDAwMC03ZmZmOTc1NTAwMDAgci0tcCAwMDAwMDAwMCBm ZDowMCAxNjc5ODkyOSAgICAgICAgICAgICAgICAgICAvdXNyL2xpYi9sb2NhbGUvZW5fVVMu dXRmOC9MQ19USU1FCjdmZmY5NzU1MDAwMC03ZmZmOTc2ODAwMDAgci0tcCAwMDAwMDAwMCBm ZDowMCAyNTE2NzkyNCAgICAgICAgICAgICAgICAgICAvdXNyL2xpYi9sb2NhbGUvZW5fVVMu dXRmOC9MQ19DT0xMQVRFCjdmZmY5NzY4MDAwMC03ZmZmOTc2OTAwMDAgci0tcCAwMDAwMDAw MCBmZDowMCAxNjc5ODkyNyAgICAgICAgICAgICAgICAgICAvdXNyL2xpYi9sb2NhbGUvZW5f VVMudXRmOC9MQ19NT05FVEFSWQo3ZmZmOTc2OTAwMDAtN2ZmZjk3NmEwMDAwIHItLXAgMDAw MDAwMDAgZmQ6MDAgMjUxMSAgICAgICAgICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxl L2VuX1VTLnV0ZjgvTENfTUVTU0FHRVMvU1lTX0xDX01FU1NBR0VTCjdmZmY5NzZhMDAwMC03 ZmZmOTc2YjAwMDAgci0tcCAwMDAwMDAwMCBmZDowMCAxNjc5ODk0MiAgICAgICAgICAgICAg ICAgICAvdXNyL2xpYi9sb2NhbGUvZW5fVVMudXRmOC9MQ19QQVBFUgo3ZmZmOTc2YjAwMDAt N2ZmZjk3NmMwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMjUxNjc5MjcgICAgICAgICAgICAg ICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfTkFNRQo3ZmZmOTc2YzAwMDAt N2ZmZjk3NmQwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5MjQgICAgICAgICAgICAg ICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfQUREUkVTUwo3ZmZmOTc2ZDAw MDAtN2ZmZjk3NmUwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5MjggICAgICAgICAg ICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfVEVMRVBIT05FCjdmZmY5 NzZlMDAwMC03ZmZmOTc2ZjAwMDAgci0tcCAwMDAwMDAwMCBmZDowMCAxNjc5ODkyNiAgICAg ICAgICAgICAgICAgICAvdXNyL2xpYi9sb2NhbGUvZW5fVVMudXRmOC9MQ19NRUFTVVJFTUVO VAo3ZmZmOTc2ZjAwMDAtN2ZmZjk3NzAwMDAwIHItLXMgMDAwMDAwMDAgZmQ6MDAgODM5MDY2 OSAgICAgICAgICAgICAgICAgICAgL3Vzci9saWI2NC9nY29udi9nY29udi1tb2R1bGVzLmNh Y2hlCjdmZmY5NzcwMDAwMC03ZmZmOTc5MzAwMDAgci14cCAwMDAwMDAwMCBmZDowMCAyNTE2 NzkzNiAgICAgICAgICAgICAgICAgICAvdXNyL2xpYjY0L3Bvd2VyOC9saWJjLTIuMjUuc28K N2ZmZjk3OTMwMDAwLTdmZmY5Nzk0MDAwMCAtLS1wIDAwMjMwMDAwIGZkOjAwIDI1MTY3OTM2 ICAgICAgICAgICAgICAgICAgIC91c3IvbGliNjQvcG93ZXI4L2xpYmMtMi4yNS5zbwo3ZmZm OTc5NDAwMDAtN2ZmZjk3OTUwMDAwIHItLXAgMDAyMzAwMDAgZmQ6MDAgMjUxNjc5MzYgICAg ICAgICAgICAgICAgICAgL3Vzci9saWI2NC9wb3dlcjgvbGliYy0yLjI1LnNvCjdmZmY5Nzk1 MDAwMC03ZmZmOTc5NjAwMDAgcnctcCAwMDI0MDAwMCBmZDowMCAyNTE2NzkzNiAgICAgICAg ICAgICAgICAgICAvdXNyL2xpYjY0L3Bvd2VyOC9saWJjLTIuMjUuc28KN2ZmZjk3OTYwMDAw LTdmZmY5Nzk3MDAwMCByLS1wIDAwMDAwMDAwIGZkOjAwIDE2Nzk4OTI1ICAgICAgICAgICAg ICAgICAgIC91c3IvbGliL2xvY2FsZS9lbl9VUy51dGY4L0xDX0lERU5USUZJQ0FUSU9OCjdm ZmY5Nzk3MDAwMC03ZmZmOTc5OTAwMDAgci14cCAwMDAwMDAwMCAwMDowMCAwICAgICAgICAg ICAgICAgICAgICAgICAgICBbdmRzb10KN2ZmZjk3OTkwMDAwLTdmZmY5NzlkMDAwMCByLXhw IDAwMDAwMDAwIGZkOjAwIDgzOTAzMjkgICAgICAgICAgICAgICAgICAgIC91c3IvbGliNjQv bGQtMi4yNS5zbwo3ZmZmOTc5ZDAwMDAtN2ZmZjk3OWUwMDAwIHItLXAgMDAwMzAwMDAgZmQ6 MDAgODM5MDMyOSAgICAgICAgICAgICAgICAgICAgL3Vzci9saWI2NC9sZC0yLjI1LnNvCjdm ZmY5NzllMDAwMC03ZmZmOTc5ZjAwMDAgcnctcCAwMDA0MDAwMCBmZDowMCA4MzkwMzI5ICAg ICAgICAgICAgICAgICAgICAvdXNyL2xpYjY0L2xkLTIuMjUuc28KN2ZmZmRkMTEwMDAwLTdm ZmZkZDE0MDAwMCBydy1wIDAwMDAwMDAwIDAwOjAwIDAgICAgICAgICAgICAgICAgICAgICAg ICAgIFtzdGFja10K --------------B8C2898AF8C15A8B4627E032-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f71.google.com (mail-pg0-f71.google.com [74.125.83.71]) by kanga.kvack.org (Postfix) with ESMTP id 461C36B0033 for ; Sun, 5 Nov 2017 07:54:56 -0500 (EST) Received: by mail-pg0-f71.google.com with SMTP id 15so9271395pgc.16 for ; Sun, 05 Nov 2017 04:54:56 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id t67sor2824349pgb.205.2017.11.05.04.54.54 for (Google Transport Security); Sun, 05 Nov 2017 04:54:55 -0800 (PST) Date: Sun, 5 Nov 2017 23:54:43 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171105235443.045fb4b7@roar.ozlabs.ibm.com> In-Reply-To: <919a1cb5-c3b5-ddee-d6a6-0994c282ae84@redhat.com> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <919a1cb5-c3b5-ddee-d6a6-0994c282ae84@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer Cc: "Aneesh Kumar K . V" , linuxppc-dev@lists.ozlabs.org, linux-mm On Sun, 5 Nov 2017 13:35:40 +0100 Florian Weimer wrote: > On 11/05/2017 01:18 PM, Nicholas Piggin wrote: > > > There was a recent change to move to 128TB address space by default, > > and option for 512TB addresses if explicitly requested. > > Do you have a commit hash for the introduction of 128TB by default? Thanks. I guess this one f6eedbba7a26 ("powerpc/mm/hash: Increase VA range to 128TB") > > > Your brk request asked for > 128TB which the kernel gave it, but the > > address limit in the paca that the SLB miss tests against was not > > updated to reflect the switch to 512TB address space. > > > > Why is your brk starting so high? Are you trying to test the > 128TB > > case, or maybe something is confused by the 64->128TB change? What's > > the strace look like if you run on a distro or <= 4.10 kernel? > > I think it is a consequence of running with an explicit loader > invocation. With that, the heap is placed above ld.so, which can be > quite high in the address space. > > I'm attaching two runs of cat, one executing directly as /bin/cat, and > one with /lib64/ld64.so.1 /bin/cat. > > Fortunately, this does *not* apply to PIE binaries (also attached). > However, explicit loader invocations are sometimes used in test suites > (not just for glibc), and these sporadic test failures are quite annoying. > > Do you still need the strace log? And if yes, of what exactly? Thanks, that should be quite helpful. I'll spend a bit more time to study it, I'll let you know if I need any other traces. > > > Something like the following patch may help if you could test. > > Okay, this will take some time. It's no rush, there will probably be a revision to come. Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f69.google.com (mail-oi0-f69.google.com [209.85.218.69]) by kanga.kvack.org (Postfix) with ESMTP id 501606B0033 for ; Sun, 5 Nov 2017 09:50:33 -0500 (EST) Received: by mail-oi0-f69.google.com with SMTP id e123so7854848oig.14 for ; Sun, 05 Nov 2017 06:50:33 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id c3si4714189oia.140.2017.11.05.06.50.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 05 Nov 2017 06:50:32 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> From: Florian Weimer Message-ID: <6d293bb3-78bd-6d20-3684-be6358bd3d7b@redhat.com> Date: Sun, 5 Nov 2017 15:50:28 +0100 MIME-Version: 1.0 In-Reply-To: <20171105231850.5e313e46@roar.ozlabs.ibm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin , "Aneesh Kumar K . V" Cc: linuxppc-dev@lists.ozlabs.org, linux-mm On 11/05/2017 01:18 PM, Nicholas Piggin wrote: > Something like the following patch may help if you could test. The patch appears to fix it: # /lib64/ld64.so.1 ./a.out initial brk value: 0x7fffe4590000 probing at 0x80000001fffc I used the follow simplified reproducer: #include #include #include #include #include int main (void) { errno = 0; void *p = sbrk (0); if (errno != 0) err (1, "sbrk (0)"); printf ("initial brk value: %p\n", p); unsigned long long target = 0x800000020000ULL; if ((uintptr_t) p >= target) errx (1, "initial brk value is already above target"); unsigned long long increment = target - (uintptr_t) p; errno = 0; sbrk (increment); if (errno != 0) err (1, "sbrk (0x%llx)", increment); volatile int *pi = (volatile int *) (target - 4); printf ("probing at %p\n", pi); *pi = 1; } It is still probabilistic because if the increment is too large, the second sbrk call will fail with an out of memory error (which is expected), so you'll have to run it a couple of times. If the test fails, the write at the will segfault. Thanks, Florian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200]) by kanga.kvack.org (Postfix) with ESMTP id ABC9C6B0038 for ; Mon, 6 Nov 2017 01:18:18 -0500 (EST) Received: by mail-wr0-f200.google.com with SMTP id v105so5451072wrc.11 for ; Sun, 05 Nov 2017 22:18:18 -0800 (PST) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com. [148.163.158.5]) by mx.google.com with ESMTPS id f2si76662edc.211.2017.11.05.22.18.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 05 Nov 2017 22:18:17 -0800 (PST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id vA66ERDV127233 for ; Mon, 6 Nov 2017 01:18:15 -0500 Received: from e06smtp12.uk.ibm.com (e06smtp12.uk.ibm.com [195.75.94.108]) by mx0a-001b2d01.pphosted.com with ESMTP id 2e26wadfvq-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 06 Nov 2017 01:18:13 -0500 Received: from localhost by e06smtp12.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 6 Nov 2017 06:18:12 -0000 From: "Aneesh Kumar K.V" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory In-Reply-To: <20171105231850.5e313e46@roar.ozlabs.ibm.com> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> Date: Mon, 06 Nov 2017 11:48:06 +0530 MIME-Version: 1.0 Content-Type: text/plain Message-Id: <871slcszfl.fsf@linux.vnet.ibm.com> Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin , Florian Weimer , "Kirill A. Shutemov" Cc: linuxppc-dev@lists.ozlabs.org, linux-mm Nicholas Piggin writes: > On Fri, 3 Nov 2017 18:05:20 +0100 > Florian Weimer wrote: > >> We are seeing an issue on ppc64le and ppc64 (and perhaps on some arm >> variant, but I have not seen it on our own builders) where running >> localedef as part of the glibc build crashes with a segmentation fault. >> >> Kernel version is 4.13.9 (Fedora 26 variant). >> >> I have only seen this with an explicit loader invocation, like this: >> >> while I18NPATH=. /lib64/ld64.so.1 /usr/bin/localedef >> --alias-file=../intl/locale.alias --no-archive -i locales/nl_AW -c -f >> charmaps/UTF-8 >> --prefix=/builddir/build/BUILDROOT/glibc-2.26-16.fc27.ppc64 nl_AW ; do : >> ; done >> >> To be run in the localedata subdirectory of a glibc *source* tree, after >> a build. You may have to create the >> /builddir/build/BUILDROOT/glibc-2.26-16.fc27.ppc64/usr/lib/locale >> directory. I have only reproduced this inside a Fedora 27 chroot on a >> Fedora 26 host, but there it does not matter if you run the old (chroot) >> or newly built binary. >> >> I filed this as a glibc bug for tracking: >> >> https://sourceware.org/bugzilla/show_bug.cgi?id=22390 >> >> There's an strace log and a coredump from the crash. >> >> I think the data shows that the address in question should be writable. >> >> The crossed 0x0000800000000000 binary is very suggestive. I think that >> based on the operation of glibc's malloc, this write would be the first >> time this happens during the lifetime of the process. >> >> Does that ring any bells? Is there anything I can do to provide more >> data? The host is an LPAR with a stock Fedora 26 kernel, so I can use >> any diagnostics tool which is provided by Fedora. > > There was a recent change to move to 128TB address space by default, > and option for 512TB addresses if explicitly requested. > > Your brk request asked for > 128TB which the kernel gave it, but the > address limit in the paca that the SLB miss tests against was not > updated to reflect the switch to 512TB address space. We should not return that address, unless we requested with a hint value of > 128TB. IIRC we discussed this early during the mmap interface change and said, we will return an address > 128T only if the hint address is above 128TB (not hint addr + length). I am not sure why we are finding us returning and address > 128TB with paca limit set to 128TB? > > Why is your brk starting so high? Are you trying to test the > 128TB > case, or maybe something is confused by the 64->128TB change? What's > the strace look like if you run on a distro or <= 4.10 kernel? > > Something like the following patch may help if you could test. > > Thanks, > Nick > -aneesh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f70.google.com (mail-pg0-f70.google.com [74.125.83.70]) by kanga.kvack.org (Postfix) with ESMTP id F2C9E6B0038 for ; Mon, 6 Nov 2017 01:48:08 -0500 (EST) Received: by mail-pg0-f70.google.com with SMTP id 191so11753934pgd.0 for ; Sun, 05 Nov 2017 22:48:08 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id y1sor3575809pfl.26.2017.11.05.22.48.07 for (Google Transport Security); Sun, 05 Nov 2017 22:48:07 -0800 (PST) Date: Mon, 6 Nov 2017 17:47:52 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171106174707.19f6c495@roar.ozlabs.ibm.com> In-Reply-To: <871slcszfl.fsf@linux.vnet.ibm.com> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "Aneesh Kumar K.V" Cc: Florian Weimer , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm On Mon, 06 Nov 2017 11:48:06 +0530 "Aneesh Kumar K.V" wrote: > Nicholas Piggin writes: > > > On Fri, 3 Nov 2017 18:05:20 +0100 > > Florian Weimer wrote: > > > >> We are seeing an issue on ppc64le and ppc64 (and perhaps on some arm > >> variant, but I have not seen it on our own builders) where running > >> localedef as part of the glibc build crashes with a segmentation fault. > >> > >> Kernel version is 4.13.9 (Fedora 26 variant). > >> > >> I have only seen this with an explicit loader invocation, like this: > >> > >> while I18NPATH=. /lib64/ld64.so.1 /usr/bin/localedef > >> --alias-file=../intl/locale.alias --no-archive -i locales/nl_AW -c -f > >> charmaps/UTF-8 > >> --prefix=/builddir/build/BUILDROOT/glibc-2.26-16.fc27.ppc64 nl_AW ; do : > >> ; done > >> > >> To be run in the localedata subdirectory of a glibc *source* tree, after > >> a build. You may have to create the > >> /builddir/build/BUILDROOT/glibc-2.26-16.fc27.ppc64/usr/lib/locale > >> directory. I have only reproduced this inside a Fedora 27 chroot on a > >> Fedora 26 host, but there it does not matter if you run the old (chroot) > >> or newly built binary. > >> > >> I filed this as a glibc bug for tracking: > >> > >> https://sourceware.org/bugzilla/show_bug.cgi?id=22390 > >> > >> There's an strace log and a coredump from the crash. > >> > >> I think the data shows that the address in question should be writable. > >> > >> The crossed 0x0000800000000000 binary is very suggestive. I think that > >> based on the operation of glibc's malloc, this write would be the first > >> time this happens during the lifetime of the process. > >> > >> Does that ring any bells? Is there anything I can do to provide more > >> data? The host is an LPAR with a stock Fedora 26 kernel, so I can use > >> any diagnostics tool which is provided by Fedora. > > > > There was a recent change to move to 128TB address space by default, > > and option for 512TB addresses if explicitly requested. > > > > Your brk request asked for > 128TB which the kernel gave it, but the > > address limit in the paca that the SLB miss tests against was not > > updated to reflect the switch to 512TB address space. > > We should not return that address, unless we requested with a hint value > of > 128TB. IIRC we discussed this early during the mmap interface > change and said, we will return an address > 128T only if the hint > address is above 128TB (not hint addr + length). Yeah, I'm thinking we should change that. Make explicit addr + length hint return > 128TB. Well, it already seems to for this case, which is why powerpc breaks. This restriction was added for reasonably well written apps that just happened to assume they don't get > 128TB va returned by mmap. An app which asked for addr < 128TB && addr + len > 128TB and were relying on that to fail is very different. I don't think we should add a big unintuitive wart to the interface for such an obscure and broken type of app. "You get < 128TB unless explicitly requested." Simple, reasonable, obvious rule. Avoids breaking apps that store some bits in the top of pointers (provided that memory allocator userspace libraries also do the right thing). Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f71.google.com (mail-oi0-f71.google.com [209.85.218.71]) by kanga.kvack.org (Postfix) with ESMTP id 09AB16B0253 for ; Mon, 6 Nov 2017 03:10:16 -0500 (EST) Received: by mail-oi0-f71.google.com with SMTP id s185so9474804oif.16 for ; Mon, 06 Nov 2017 00:10:16 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id a2si1356261otd.64.2017.11.06.00.10.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Nov 2017 00:10:15 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> From: Florian Weimer Message-ID: Date: Mon, 6 Nov 2017 09:10:10 +0100 MIME-Version: 1.0 In-Reply-To: <871slcszfl.fsf@linux.vnet.ibm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "Aneesh Kumar K.V" , Nicholas Piggin , "Kirill A. Shutemov" Cc: linuxppc-dev@lists.ozlabs.org, linux-mm On 11/06/2017 07:18 AM, Aneesh Kumar K.V wrote: > We should not return that address, unless we requested with a hint value > of > 128TB. IIRC we discussed this early during the mmap interface > change and said, we will return an address > 128T only if the hint > address is above 128TB (not hint addr + length). I am not sure why > we are finding us returning and address > 128TB with paca limit set to > 128TB? See the memory maps I posted. I think it was not anticipated that the heap could be near the 128 TiB limit because it is placed next to the initially mapped object. I think this could become worse once we have static PIE support because static PIE binaries likely have the same memory layout. (Ordinary PIE does not.) Thanks, Florian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f70.google.com (mail-oi0-f70.google.com [209.85.218.70]) by kanga.kvack.org (Postfix) with ESMTP id 0F5D16B0253 for ; Mon, 6 Nov 2017 03:11:41 -0500 (EST) Received: by mail-oi0-f70.google.com with SMTP id 14so9605911oii.2 for ; Mon, 06 Nov 2017 00:11:41 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id z7si1222502otb.330.2017.11.06.00.11.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Nov 2017 00:11:40 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> From: Florian Weimer Message-ID: <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> Date: Mon, 6 Nov 2017 09:11:37 +0100 MIME-Version: 1.0 In-Reply-To: <20171106174707.19f6c495@roar.ozlabs.ibm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin , "Aneesh Kumar K.V" Cc: "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm On 11/06/2017 07:47 AM, Nicholas Piggin wrote: > "You get < 128TB unless explicitly requested." > > Simple, reasonable, obvious rule. Avoids breaking apps that store > some bits in the top of pointers (provided that memory allocator > userspace libraries also do the right thing). So brk would simplify fail instead of crossing the 128 TiB threshold? glibc malloc should cope with that and switch to malloc, but this code path is obviously less well-tested than the regular way. Thanks, Florian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f69.google.com (mail-pg0-f69.google.com [74.125.83.69]) by kanga.kvack.org (Postfix) with ESMTP id 659046B0038 for ; Mon, 6 Nov 2017 03:25:37 -0500 (EST) Received: by mail-pg0-f69.google.com with SMTP id k7so11958220pga.8 for ; Mon, 06 Nov 2017 00:25:37 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id k8sor3348920pgo.33.2017.11.06.00.25.36 for (Google Transport Security); Mon, 06 Nov 2017 00:25:36 -0800 (PST) Date: Mon, 6 Nov 2017 19:25:24 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171106192524.12ea3187@roar.ozlabs.ibm.com> In-Reply-To: <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer Cc: "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm On Mon, 6 Nov 2017 09:11:37 +0100 Florian Weimer wrote: > On 11/06/2017 07:47 AM, Nicholas Piggin wrote: > > "You get < 128TB unless explicitly requested." > > > > Simple, reasonable, obvious rule. Avoids breaking apps that store > > some bits in the top of pointers (provided that memory allocator > > userspace libraries also do the right thing). > > So brk would simplify fail instead of crossing the 128 TiB threshold? Yes, that was the intention and that's what x86 seems to do. > > glibc malloc should cope with that and switch to malloc, but this code > path is obviously less well-tested than the regular way. Switch to mmap() I guess you meant? powerpc has a couple of bugs in corner cases, so those should be fixed according to intended policy for stable kernels I think. But I question the policy. Just seems like an ugly and ineffective wart. Exactly for such cases as this -- behaviour would change from run to run depending on your address space randomization for example! In case your brk happens to land nicely on 128TB then the next one would succeed. If not, then behaviour changes. And you didn't catch or prevent any bugs. Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f72.google.com (mail-oi0-f72.google.com [209.85.218.72]) by kanga.kvack.org (Postfix) with ESMTP id A69916B025F for ; Mon, 6 Nov 2017 03:32:35 -0500 (EST) Received: by mail-oi0-f72.google.com with SMTP id m198so9583745oig.20 for ; Mon, 06 Nov 2017 00:32:35 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id l125si5218884oib.84.2017.11.06.00.32.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Nov 2017 00:32:34 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> From: Florian Weimer Message-ID: <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> Date: Mon, 6 Nov 2017 09:32:25 +0100 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "Aneesh Kumar K.V" , Nicholas Piggin Cc: "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm On 11/06/2017 09:30 AM, Aneesh Kumar K.V wrote: > On 11/06/2017 01:55 PM, Nicholas Piggin wrote: >> On Mon, 6 Nov 2017 09:11:37 +0100 >> Florian Weimer wrote: >> >>> On 11/06/2017 07:47 AM, Nicholas Piggin wrote: >>>> "You get < 128TB unless explicitly requested." >>>> >>>> Simple, reasonable, obvious rule. Avoids breaking apps that store >>>> some bits in the top of pointers (provided that memory allocator >>>> userspace libraries also do the right thing). >>> >>> So brk would simplify fail instead of crossing the 128 TiB threshold? >> >> Yes, that was the intention and that's what x86 seems to do. >> >>> >>> glibc malloc should cope with that and switch to malloc, but this code >>> path is obviously less well-tested than the regular way. >> >> Switch to mmap() I guess you meant? Yes, sorry. >> powerpc has a couple of bugs in corner cases, so those should be fixed >> according to intended policy for stable kernels I think. >> >> But I question the policy. Just seems like an ugly and ineffective wart. >> Exactly for such cases as this -- behaviour would change from run to run >> depending on your address space randomization for example! In case your >> brk happens to land nicely on 128TB then the next one would succeed. > > Why ? It should not change between run to run. We limit the free > area search range based on hint address. So we should get consistent > results across run. even if we changed the context.addr_limit. The size of the gap to the 128 TiB limit varies between runs because of ASLR. So some runs would use brk alone, others would use brk + malloc. That's not really desirable IMHO. Thanks, Florian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f69.google.com (mail-pg0-f69.google.com [74.125.83.69]) by kanga.kvack.org (Postfix) with ESMTP id D606A6B026C for ; Mon, 6 Nov 2017 05:20:50 -0500 (EST) Received: by mail-pg0-f69.google.com with SMTP id j3so12356377pga.5 for ; Mon, 06 Nov 2017 02:20:50 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id f79sor3673222pfk.29.2017.11.06.02.20.49 for (Google Transport Security); Mon, 06 Nov 2017 02:20:49 -0800 (PST) Date: Mon, 6 Nov 2017 21:20:38 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171106212038.61163712@roar.ozlabs.ibm.com> In-Reply-To: <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer Cc: "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm On Mon, 6 Nov 2017 09:32:25 +0100 Florian Weimer wrote: > On 11/06/2017 09:30 AM, Aneesh Kumar K.V wrote: > > On 11/06/2017 01:55 PM, Nicholas Piggin wrote: > >> On Mon, 6 Nov 2017 09:11:37 +0100 > >> Florian Weimer wrote: > >> > >>> On 11/06/2017 07:47 AM, Nicholas Piggin wrote: > >>>> "You get < 128TB unless explicitly requested." > >>>> > >>>> Simple, reasonable, obvious rule. Avoids breaking apps that store > >>>> some bits in the top of pointers (provided that memory allocator > >>>> userspace libraries also do the right thing). > >>> > >>> So brk would simplify fail instead of crossing the 128 TiB threshold? > >> > >> Yes, that was the intention and that's what x86 seems to do. > >> > >>> > >>> glibc malloc should cope with that and switch to malloc, but this code > >>> path is obviously less well-tested than the regular way. > >> > >> Switch to mmap() I guess you meant? > > Yes, sorry. > > >> powerpc has a couple of bugs in corner cases, so those should be fixed > >> according to intended policy for stable kernels I think. > >> > >> But I question the policy. Just seems like an ugly and ineffective wart. > >> Exactly for such cases as this -- behaviour would change from run to run > >> depending on your address space randomization for example! In case your > >> brk happens to land nicely on 128TB then the next one would succeed. > > > > Why ? It should not change between run to run. We limit the free > > area search range based on hint address. So we should get consistent > > results across run. even if we changed the context.addr_limit. > > The size of the gap to the 128 TiB limit varies between runs because of > ASLR. So some runs would use brk alone, others would use brk + malloc. > That's not really desirable IMHO. Yeah. Actually I looked at the code a bit more, and it seems that the intention is for MAP_FIXED to do exactly what I wanted. brk() uses MAP_FIXED under the covers, so this case should be okay I think. I'm just slightly happier now, but I still think it's not the right thing to do to fail an explicit request for crossing 128TB with a hint. Same fundamental criticism still applies -- it does not really solve bugs and just adds an unintuitive wart to the API, and a random change in behaviour based on randomization. Anyway I sent some patches that are split up better and hopefully solve some bugs for powerpc without changing intended policy. That's left for another discussion. Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id C58756B027C for ; Tue, 7 Nov 2017 00:07:20 -0500 (EST) Received: by mail-pf0-f199.google.com with SMTP id p2so13258926pfk.13 for ; Mon, 06 Nov 2017 21:07:20 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n3sor84703pgt.340.2017.11.06.21.07.18 for (Google Transport Security); Mon, 06 Nov 2017 21:07:19 -0800 (PST) Date: Tue, 7 Nov 2017 16:07:05 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107160705.059e0c2b@roar.ozlabs.ibm.com> In-Reply-To: <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer Cc: "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List C'ing everyone who was on the x86 56-bit user virtual address patch. I think we need more time to discuss this behaviour, in light of the regression Florian uncovered. I would propose we turn off the 56-bit user virtual address support for x86 for 4.14, and powerpc would follow and turn off its 512T support until we can get a better handle on the problems. (Actually Florian initially hit a couple of bugs in powerpc implementation, but pulling that string uncovers a whole lot of difficulties.) The bi-modal behavior switched based on a combination of mmap address hint and MAP_FIXED just sucks. It's segregating our VA space with some non-standard heuristics, and it doesn't seem to work very well. What are we trying to do? Allow SAP HANA etc use huge address spaces by coding to these specific mmap heuristics we're going to add, rather than solving it properly in a way that requires adding a new syscall or personality or prctl or sysctl. Okay, but the cost is that despite best efforts, it still changes ABI behaviour for existing applications and these heuristics will become baked into the ABI that we will have to support. Not a good tradeoff IMO. First of all, using addr and MAP_FIXED to develop our heuristic can never really give unchanged ABI. It's an in-band signal. brk() is a good example that steadily keeps incrementing address, so depending on malloc usage and address space randomization, you will get a brk() that ends exactly at 128T, then the next one will be > DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. Second, the kernel can never completely solve the problem this way. How do we know a malloc library will not ask for > 128TB addresses and pass them to an unknowing application? And lastly, there are a fair few bugs and places where description in changelogs and mailing lists does not match code. You don't want to know the mess in powerpc, but even x86 has two I can see: MAP_FIXED succeeds even when crossing 128TB addresses (where changelog indicated it should not), arch_get_unmapped_area_topdown() with an address hint is checking against TASK_SIZE rather than the limited 128TB address, so it looks like it won't follow the heuristics. So unless everyone else thinks I'm crazy and disagrees, I'd ask for a bit more time to make sure we get this interface right. I would hope for something like prctl PR_SET_MM which can be used to set our user virtual address bits on a fine grained basis. Maybe a sysctl, maybe a personality. Something out-of-band. I don't wan to get too far into that discussion yet. First we need to agree whether or not the code in the tree today is a problem. Thanks, Nick On Mon, 6 Nov 2017 09:32:25 +0100 Florian Weimer wrote: > On 11/06/2017 09:30 AM, Aneesh Kumar K.V wrote: > > On 11/06/2017 01:55 PM, Nicholas Piggin wrote: > >> On Mon, 6 Nov 2017 09:11:37 +0100 > >> Florian Weimer wrote: > >> > >>> On 11/06/2017 07:47 AM, Nicholas Piggin wrote: > >>>> "You get < 128TB unless explicitly requested." > >>>> > >>>> Simple, reasonable, obvious rule. Avoids breaking apps that store > >>>> some bits in the top of pointers (provided that memory allocator > >>>> userspace libraries also do the right thing). > >>> > >>> So brk would simplify fail instead of crossing the 128 TiB threshold? > >> > >> Yes, that was the intention and that's what x86 seems to do. > >> > >>> > >>> glibc malloc should cope with that and switch to malloc, but this code > >>> path is obviously less well-tested than the regular way. > >> > >> Switch to mmap() I guess you meant? > > Yes, sorry. > > >> powerpc has a couple of bugs in corner cases, so those should be fixed > >> according to intended policy for stable kernels I think. > >> > >> But I question the policy. Just seems like an ugly and ineffective wart. > >> Exactly for such cases as this -- behaviour would change from run to run > >> depending on your address space randomization for example! In case your > >> brk happens to land nicely on 128TB then the next one would succeed. > > > > Why ? It should not change between run to run. We limit the free > > area search range based on hint address. So we should get consistent > > results across run. even if we changed the context.addr_limit. > > The size of the gap to the 128 TiB limit varies between runs because of > ASLR. So some runs would use brk alone, others would use brk + malloc. > That's not really desirable IMHO. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f71.google.com (mail-oi0-f71.google.com [209.85.218.71]) by kanga.kvack.org (Postfix) with ESMTP id 25BAB6B0288 for ; Tue, 7 Nov 2017 03:15:28 -0500 (EST) Received: by mail-oi0-f71.google.com with SMTP id o126so7689891oif.21 for ; Tue, 07 Nov 2017 00:15:28 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id f20si279936oti.4.2017.11.07.00.15.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Nov 2017 00:15:27 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> From: Florian Weimer Message-ID: Date: Tue, 7 Nov 2017 09:15:21 +0100 MIME-Version: 1.0 In-Reply-To: <20171107160705.059e0c2b@roar.ozlabs.ibm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin Cc: "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On 11/07/2017 06:07 AM, Nicholas Piggin wrote: > First of all, using addr and MAP_FIXED to develop our heuristic can > never really give unchanged ABI. It's an in-band signal. brk() is a > good example that steadily keeps incrementing address, so depending > on malloc usage and address space randomization, you will get a brk() > that ends exactly at 128T, then the next one will be > > DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. Note that this brk phenomenon is only a concern for some currently obscure process memory layouts where the heap ends up at the top of the address space. Usually, there is something above it which eliminates the possibility that it can cross into the 128 TiB wilderness. So the brk problem only happens on some architectures (e.g., not x86-64), and only with strange ways of running programs (explicitly ld.so invocation and likely static PIE, too). > So unless everyone else thinks I'm crazy and disagrees, I'd ask for > a bit more time to make sure we get this interface right. I would > hope for something like prctl PR_SET_MM which can be used to set > our user virtual address bits on a fine grained basis. Maybe a > sysctl, maybe a personality. Something out-of-band. I don't wan to > get too far into that discussion yet. First we need to agree whether > or not the code in the tree today is a problem. There is certainly more demand for similar functionality, like creating mappings below 2 GB/4 GB/32 GB, and probably other bit patterns. Hotspot would use this to place the heap with compressed oops, instead of manually hunting for a suitable place for the mapping. (Essentially, 32-bit pointers on 64-bit architectures for sufficiently small heap sizes.) It would perhaps be possible to use the hints address as a source of the bit count, for full flexibility. And the mapping should be placed into the upper half of the selected window if possible. MAP_FIXED is near-impossible to use correctly. I hope you don't expect applications to do that. If you want address-based opt in, it should work without MAP_FIXED. Sure, in obscure cases, applications might still see out-of-range addresses, but I expected a full opt-out based on RLIMIT_AS would be sufficient for them. Thanks, Florian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f200.google.com (mail-pf0-f200.google.com [209.85.192.200]) by kanga.kvack.org (Postfix) with ESMTP id 519BF280245 for ; Tue, 7 Nov 2017 04:25:07 -0500 (EST) Received: by mail-pf0-f200.google.com with SMTP id t188so13876637pfd.20 for ; Tue, 07 Nov 2017 01:25:07 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id g6sor246194pfb.121.2017.11.07.01.25.06 for (Google Transport Security); Tue, 07 Nov 2017 01:25:06 -0800 (PST) Date: Tue, 7 Nov 2017 20:24:49 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107202449.168dce9e@roar.ozlabs.ibm.com> In-Reply-To: References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer Cc: "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, 7 Nov 2017 09:15:21 +0100 Florian Weimer wrote: > On 11/07/2017 06:07 AM, Nicholas Piggin wrote: > > > First of all, using addr and MAP_FIXED to develop our heuristic can > > never really give unchanged ABI. It's an in-band signal. brk() is a > > good example that steadily keeps incrementing address, so depending > > on malloc usage and address space randomization, you will get a brk() > > that ends exactly at 128T, then the next one will be > > > DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. > > Note that this brk phenomenon is only a concern for some currently > obscure process memory layouts where the heap ends up at the top of the > address space. Usually, there is something above it which eliminates > the possibility that it can cross into the 128 TiB wilderness. So the > brk problem only happens on some architectures (e.g., not x86-64), and > only with strange ways of running programs (explicitly ld.so invocation > and likely static PIE, too). That's true, but there was an ABI change and the result is that it changed behaviour. And actually if it were not for a powerpc bug that caused a segfault, the allocation would have worked, and you probably would never have filed the bug. However the program would have been given a pointer > 128TB, which is what we were trying to avoid -- if your app also put metadata in the top of the pointers, it would have broken. So, are any obscure apps that could break? I don't know. Maybe it's quite unlikely. Is that enough to go ahead with changing behaviour? I'm not going to put up a big fight about this if others feel it's not a problem. I raise it because in debugging this and seeing the changed behaviour, the differences between implementations (x86, powerpc hash, and powerpc radix, all have different edge case behaviour), and that none of them seem to conform exactly to the heuristics described in the changelogs, and there seems to be no man page updates that I can see, it raises some red flags. So I'm calling for one last opportunity to 1) agree on the desired behaviour, and 2) ensure implementations are conforming. > > So unless everyone else thinks I'm crazy and disagrees, I'd ask for > > a bit more time to make sure we get this interface right. I would > > hope for something like prctl PR_SET_MM which can be used to set > > our user virtual address bits on a fine grained basis. Maybe a > > sysctl, maybe a personality. Something out-of-band. I don't wan to > > get too far into that discussion yet. First we need to agree whether > > or not the code in the tree today is a problem. > > There is certainly more demand for similar functionality, like creating > mappings below 2 GB/4 GB/32 GB, and probably other bit patterns. > Hotspot would use this to place the heap with compressed oops, instead > of manually hunting for a suitable place for the mapping. (Essentially, > 32-bit pointers on 64-bit architectures for sufficiently small heap > sizes.) It would perhaps be possible to use the hints address as a > source of the bit count, for full flexibility. And the mapping should > be placed into the upper half of the selected window if possible. > > MAP_FIXED is near-impossible to use correctly. I hope you don't expect > applications to do that. If you want address-based opt in, it should > work without MAP_FIXED. Sure, in obscure cases, applications might > still see out-of-range addresses, but I expected a full opt-out based on > RLIMIT_AS would be sufficient for them. All good points, and no I don't think MAP_FIXED is a substitute. I don't want to get too far ahead of ourselves, but I don't see why some new interfaces with reasonably flexible and extensible ways to specify VA behaviour is a bad idea. We already went through similar with the ADDR_COMPAT_LAYOUT, and some various address space randomization options, and now this, and they're all different... As you noted, for many years now with compressed pointers and data in pointers, address space bits have *mattered* to applications and we can't change that. Surely we're going to have to solve it *properly* sooner or later, why not now? We could do both existing heuristic *and* make nicer interfaces, but the >128TB support seems like a good place to do it because very few apps will have to change code, and we can leave behaviour exactly unchanged for 99.99999% that will never need so much memory. Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f198.google.com (mail-wr0-f198.google.com [209.85.128.198]) by kanga.kvack.org (Postfix) with ESMTP id 2CD72280245 for ; Tue, 7 Nov 2017 06:15:48 -0500 (EST) Received: by mail-wr0-f198.google.com with SMTP id o88so7463196wrb.18 for ; Tue, 07 Nov 2017 03:15:48 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id y14sor766345ede.42.2017.11.07.03.15.45 for (Google Transport Security); Tue, 07 Nov 2017 03:15:46 -0800 (PST) Date: Tue, 7 Nov 2017 14:15:43 +0300 From: "Kirill A. Shutemov" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171107160705.059e0c2b@roar.ozlabs.ibm.com> Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin Cc: Florian Weimer , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, Nov 07, 2017 at 04:07:05PM +1100, Nicholas Piggin wrote: > C'ing everyone who was on the x86 56-bit user virtual address patch. > > I think we need more time to discuss this behaviour, in light of the > regression Florian uncovered. I would propose we turn off the 56-bit > user virtual address support for x86 for 4.14, and powerpc would > follow and turn off its 512T support until we can get a better handle > on the problems. (Actually Florian initially hit a couple of bugs in > powerpc implementation, but pulling that string uncovers a whole lot > of difficulties.) > > The bi-modal behavior switched based on a combination of mmap address > hint and MAP_FIXED just sucks. It's segregating our VA space with > some non-standard heuristics, and it doesn't seem to work very well. > > What are we trying to do? Allow SAP HANA etc use huge address spaces > by coding to these specific mmap heuristics we're going to add, > rather than solving it properly in a way that requires adding a new > syscall or personality or prctl or sysctl. Okay, but the cost is that > despite best efforts, it still changes ABI behaviour for existing > applications and these heuristics will become baked into the ABI that > we will have to support. Not a good tradeoff IMO. > > First of all, using addr and MAP_FIXED to develop our heuristic can > never really give unchanged ABI. It's an in-band signal. brk() is a > good example that steadily keeps incrementing address, so depending > on malloc usage and address space randomization, you will get a brk() > that ends exactly at 128T, then the next one will be > > DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. No, it won't. You will hit stack first. > Second, the kernel can never completely solve the problem this way. > How do we know a malloc library will not ask for > 128TB addresses > and pass them to an unknowing application? The idea is that an application can provide hint (mallopt() ?) to malloc implementation that it's ready to full address space. In this case, malloc can use mmap((void *) -1,...) for its allocations and get full address space this way. > And lastly, there are a fair few bugs and places where description > in changelogs and mailing lists does not match code. You don't want > to know the mess in powerpc, but even x86 has two I can see: > MAP_FIXED succeeds even when crossing 128TB addresses (where changelog > indicated it should not), Hm. I don't see where the changelog indicated that MAP_FIXED across 128TB shouldn't work. My intention was that it should, although I haven't stated it in the changelog. The idea was we shouldn't allow to slip above 47-bits by accidentally. Correctly functioning program would never request addr+len above 47-bit with MAP_FIXED, unless it's ready to handle such addresses. Otherwise the request would simply fail on machine that doesn't support large VA. In contrast, addr+len above 47-bit without MAP_FIXED will not fail on machine that doesn't support large VA, kernel will find another place under 47-bit. And I can imagine a reasonable application that does something like this. So we cannot rely that application is ready to handle large addresses if we see addr+len without MAP_FIXED. > arch_get_unmapped_area_topdown() with an address hint is checking > against TASK_SIZE rather than the limited 128TB address, so it looks > like it won't follow the heuristics. You are right. This is broken. If user would request mapping above vdso, but below DEFAULT_MAP_WINDOW it will succeed. I'll send patch to fix this. But it doesn't look as a show-stopper to me. Re-checking things for this reply I found actual bug, see: http://lkml.kernel.org/r/20171107103804.47341-1-kirill.shutemov@linux.intel.com > So unless everyone else thinks I'm crazy and disagrees, I'd ask for > a bit more time to make sure we get this interface right. I would > hope for something like prctl PR_SET_MM which can be used to set > our user virtual address bits on a fine grained basis. Maybe a > sysctl, maybe a personality. Something out-of-band. I don't wan to > get too far into that discussion yet. First we need to agree whether > or not the code in the tree today is a problem. Well, we've discussed before all options you are proposing. Linus wanted a minimalistic interface, so we took this path for now. We can always add more ways to get access to full address space later. -- Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200]) by kanga.kvack.org (Postfix) with ESMTP id 4444D280256 for ; Tue, 7 Nov 2017 06:17:01 -0500 (EST) Received: by mail-wr0-f200.google.com with SMTP id o14so418792wrf.6 for ; Tue, 07 Nov 2017 03:17:01 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id v11sor719037edb.25.2017.11.07.03.16.59 for (Google Transport Security); Tue, 07 Nov 2017 03:17:00 -0800 (PST) Date: Tue, 7 Nov 2017 14:16:58 +0300 From: "Kirill A. Shutemov" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107111658.ifkbeg4dnlheabnr@node.shutemov.name> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer Cc: Nicholas Piggin , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, Nov 07, 2017 at 09:15:21AM +0100, Florian Weimer wrote: > MAP_FIXED is near-impossible to use correctly. I hope you don't expect > applications to do that. If you want address-based opt in, it should work > without MAP_FIXED. Sure, in obscure cases, applications might still see > out-of-range addresses, but I expected a full opt-out based on RLIMIT_AS > would be sufficient for them. Just use mmap(-1), without MAP_FIXED to get full address space. -- Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f71.google.com (mail-oi0-f71.google.com [209.85.218.71]) by kanga.kvack.org (Postfix) with ESMTP id CE92C6B029C for ; Tue, 7 Nov 2017 06:26:18 -0500 (EST) Received: by mail-oi0-f71.google.com with SMTP id s185so12573733oif.16 for ; Tue, 07 Nov 2017 03:26:18 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id i203si436872oib.234.2017.11.07.03.26.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Nov 2017 03:26:17 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> From: Florian Weimer Message-ID: Date: Tue, 7 Nov 2017 12:26:12 +0100 MIME-Version: 1.0 In-Reply-To: <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "Kirill A. Shutemov" , Nicholas Piggin Cc: "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On 11/07/2017 12:15 PM, Kirill A. Shutemov wrote: >> First of all, using addr and MAP_FIXED to develop our heuristic can >> never really give unchanged ABI. It's an in-band signal. brk() is a >> good example that steadily keeps incrementing address, so depending >> on malloc usage and address space randomization, you will get a brk() >> that ends exactly at 128T, then the next one will be > >> DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. > > No, it won't. You will hit stack first. That's not actually true on POWER in some cases. See the process maps I posted here: Thanks, Florian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id CB1E56B02AA for ; Tue, 7 Nov 2017 06:44:25 -0500 (EST) Received: by mail-wm0-f69.google.com with SMTP id n8so721761wmg.4 for ; Tue, 07 Nov 2017 03:44:25 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id x28sor811033eda.19.2017.11.07.03.44.24 for (Google Transport Security); Tue, 07 Nov 2017 03:44:24 -0800 (PST) Date: Tue, 7 Nov 2017 14:44:22 +0300 From: "Kirill A. Shutemov" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107114422.bgnm5k6w2zqjoazc@node.shutemov.name> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer Cc: Nicholas Piggin , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, Nov 07, 2017 at 12:26:12PM +0100, Florian Weimer wrote: > On 11/07/2017 12:15 PM, Kirill A. Shutemov wrote: > > > > First of all, using addr and MAP_FIXED to develop our heuristic can > > > never really give unchanged ABI. It's an in-band signal. brk() is a > > > good example that steadily keeps incrementing address, so depending > > > on malloc usage and address space randomization, you will get a brk() > > > that ends exactly at 128T, then the next one will be > > > > DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. > > > > No, it won't. You will hit stack first. > > That's not actually true on POWER in some cases. See the process maps I > posted here: > > Hm? I see that in all three cases the [stack] is the last mapping. Do I miss something? -- Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f198.google.com (mail-pf0-f198.google.com [209.85.192.198]) by kanga.kvack.org (Postfix) with ESMTP id AAB0B6B02B3 for ; Tue, 7 Nov 2017 06:56:56 -0500 (EST) Received: by mail-pf0-f198.google.com with SMTP id p87so14202515pfj.21 for ; Tue, 07 Nov 2017 03:56:56 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id f10sor323193plr.93.2017.11.07.03.56.55 for (Google Transport Security); Tue, 07 Nov 2017 03:56:55 -0800 (PST) Date: Tue, 7 Nov 2017 22:56:36 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> In-Reply-To: <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "Kirill A. Shutemov" Cc: Florian Weimer , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, 7 Nov 2017 14:15:43 +0300 "Kirill A. Shutemov" wrote: > On Tue, Nov 07, 2017 at 04:07:05PM +1100, Nicholas Piggin wrote: > > C'ing everyone who was on the x86 56-bit user virtual address patch. > > > > I think we need more time to discuss this behaviour, in light of the > > regression Florian uncovered. I would propose we turn off the 56-bit > > user virtual address support for x86 for 4.14, and powerpc would > > follow and turn off its 512T support until we can get a better handle > > on the problems. (Actually Florian initially hit a couple of bugs in > > powerpc implementation, but pulling that string uncovers a whole lot > > of difficulties.) > > > > The bi-modal behavior switched based on a combination of mmap address > > hint and MAP_FIXED just sucks. It's segregating our VA space with > > some non-standard heuristics, and it doesn't seem to work very well. > > > > What are we trying to do? Allow SAP HANA etc use huge address spaces > > by coding to these specific mmap heuristics we're going to add, > > rather than solving it properly in a way that requires adding a new > > syscall or personality or prctl or sysctl. Okay, but the cost is that > > despite best efforts, it still changes ABI behaviour for existing > > applications and these heuristics will become baked into the ABI that > > we will have to support. Not a good tradeoff IMO. > > > > First of all, using addr and MAP_FIXED to develop our heuristic can > > never really give unchanged ABI. It's an in-band signal. brk() is a > > good example that steadily keeps incrementing address, so depending > > on malloc usage and address space randomization, you will get a brk() > > that ends exactly at 128T, then the next one will be > > > DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. > > No, it won't. You will hit stack first. I guess so. Florian's bug didn't crash there for some reason, okay but I suppose my point about brk is not exactly where the standard heap is, but the pattern of allocations. An allocator that uses mmap for managing its address space might do the same thing, e.g., incrementally expand existing mmaps as necessary. > > Second, the kernel can never completely solve the problem this way. > > How do we know a malloc library will not ask for > 128TB addresses > > and pass them to an unknowing application? > > The idea is that an application can provide hint (mallopt() ?) to malloc > implementation that it's ready to full address space. In this case, malloc > can use mmap((void *) -1,...) for its allocations and get full address > space this way. Point is, there's nothing stopping an allocator library or runtime from asking for mmap anywhere and returning it to userspace. Do > 128TB pointers matter so much that we should add this heuristic to prevent breakage, but little enough that we can accept some rare cases getting through? Genuine question. > > > And lastly, there are a fair few bugs and places where description > > in changelogs and mailing lists does not match code. You don't want > > to know the mess in powerpc, but even x86 has two I can see: > > MAP_FIXED succeeds even when crossing 128TB addresses (where changelog > > indicated it should not), > > Hm. I don't see where the changelog indicated that MAP_FIXED across 128TB > shouldn't work. My intention was that it should, although I haven't stated > it in the changelog. To mitigate this, we are not going to allocate virtual address space above 47-bit by default. But userspace can ask for allocation from full address space by specifying hint address (with or without MAP_FIXED) above 47-bits. Yet we got 48 bit address with 47 bit address (with MAP_FIXED). > > The idea was we shouldn't allow to slip above 47-bits by accidentally. > > Correctly functioning program would never request addr+len above 47-bit > with MAP_FIXED, unless it's ready to handle such addresses. Otherwise the > request would simply fail on machine that doesn't support large VA. > > In contrast, addr+len above 47-bit without MAP_FIXED will not fail on > machine that doesn't support large VA, kernel will find another place > under 47-bit. And I can imagine a reasonable application that does > something like this. > > So we cannot rely that application is ready to handle large > addresses if we see addr+len without MAP_FIXED. By the same logic, a request for addr > 128TB without MAP_FIXED will not fail, therefore we can't rely on that either. Or an app that links to a library that attempts MAP_FIXED allocation of addr + len above 128TB might use high bits of pointer returned by that library because those are never satisfied today and the library would fall back. > > > arch_get_unmapped_area_topdown() with an address hint is checking > > against TASK_SIZE rather than the limited 128TB address, so it looks > > like it won't follow the heuristics. > > You are right. This is broken. If user would request mapping above vdso, > but below DEFAULT_MAP_WINDOW it will succeed. > > I'll send patch to fix this. But it doesn't look as a show-stopper to me. > > Re-checking things for this reply I found actual bug, see: > > http://lkml.kernel.org/r/20171107103804.47341-1-kirill.shutemov@linux.intel.com It's not a show stopper per se, and of course I don't expect code to be bug free, it's just concerning that we haven't had any kind of regression testing or observations that have checked for this. > > > So unless everyone else thinks I'm crazy and disagrees, I'd ask for > > a bit more time to make sure we get this interface right. I would > > hope for something like prctl PR_SET_MM which can be used to set > > our user virtual address bits on a fine grained basis. Maybe a > > sysctl, maybe a personality. Something out-of-band. I don't wan to > > get too far into that discussion yet. First we need to agree whether > > or not the code in the tree today is a problem. > > Well, we've discussed before all options you are proposing. > Linus wanted a minimalistic interface, so we took this path for now. > We can always add more ways to get access to full address space later. > Right, I'm just raising it again with additional justifications. I don't think it's the right way to go or has to be rushed now. Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id 4D059280245 for ; Tue, 7 Nov 2017 07:28:29 -0500 (EST) Received: by mail-wm0-f69.google.com with SMTP id t139so776899wmt.7 for ; Tue, 07 Nov 2017 04:28:29 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id w51sor803934edd.54.2017.11.07.04.28.27 for (Google Transport Security); Tue, 07 Nov 2017 04:28:27 -0800 (PST) Date: Tue, 7 Nov 2017 15:28:25 +0300 From: "Kirill A. Shutemov" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin Cc: Florian Weimer , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, Nov 07, 2017 at 10:56:36PM +1100, Nicholas Piggin wrote: > > No, it won't. You will hit stack first. > > I guess so. Florian's bug didn't crash there for some reason, okay > but I suppose my point about brk is not exactly where the standard > heap is, but the pattern of allocations. An allocator that uses > mmap for managing its address space might do the same thing, e.g., > incrementally expand existing mmaps as necessary. With MAP_FIXED? I don't think so. > > > Second, the kernel can never completely solve the problem this way. > > > How do we know a malloc library will not ask for > 128TB addresses > > > and pass them to an unknowing application? > > > > The idea is that an application can provide hint (mallopt() ?) to malloc > > implementation that it's ready to full address space. In this case, malloc > > can use mmap((void *) -1,...) for its allocations and get full address > > space this way. > > Point is, there's nothing stopping an allocator library or runtime > from asking for mmap anywhere and returning it to userspace. Right. Nobody would stop it from doing stupid things. There are many things that a library may do that application would not be happy about. > Do > 128TB pointers matter so much that we should add this heuristic > to prevent breakage, but little enough that we can accept some rare > cases getting through? Genuine question. At the end of the day what matters is if heuristic helps prevent breakage of existing userspace and doesn't stay in the way of legitimate use of full address space. So far, it looks okay to me. > > The idea was we shouldn't allow to slip above 47-bits by accidentally. > > > > Correctly functioning program would never request addr+len above 47-bit > > with MAP_FIXED, unless it's ready to handle such addresses. Otherwise the > > request would simply fail on machine that doesn't support large VA. > > > > In contrast, addr+len above 47-bit without MAP_FIXED will not fail on > > machine that doesn't support large VA, kernel will find another place > > under 47-bit. And I can imagine a reasonable application that does > > something like this. > > > > So we cannot rely that application is ready to handle large > > addresses if we see addr+len without MAP_FIXED. > > By the same logic, a request for addr > 128TB without MAP_FIXED will > not fail, therefore we can't rely on that either. > > Or an app that links to a library that attempts MAP_FIXED allocation > of addr + len above 128TB might use high bits of pointer returned by > that library because those are never satisfied today and the library > would fall back. If you want to point that it's ABI break, yes it is. But we allow ABI break as long as nobody notices. I think it's reasonable to expect that nobody relies on such corner cases. If we would find any piece of software affect by the change we would need to reconsider. -- Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f72.google.com (mail-oi0-f72.google.com [209.85.218.72]) by kanga.kvack.org (Postfix) with ESMTP id E49A06B02BF for ; Tue, 7 Nov 2017 08:05:48 -0500 (EST) Received: by mail-oi0-f72.google.com with SMTP id q4so12744297oic.12 for ; Tue, 07 Nov 2017 05:05:48 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r21si345445ote.179.2017.11.07.05.05.47 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Nov 2017 05:05:47 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107114422.bgnm5k6w2zqjoazc@node.shutemov.name> From: Florian Weimer Message-ID: <7fc1641b-361c-2ee2-c510-f7c64d173bf8@redhat.com> Date: Tue, 7 Nov 2017 14:05:42 +0100 MIME-Version: 1.0 In-Reply-To: <20171107114422.bgnm5k6w2zqjoazc@node.shutemov.name> Content-Type: multipart/mixed; boundary="------------381B9005558AE185EB668BF2" Content-Language: en-US Sender: owner-linux-mm@kvack.org List-ID: To: "Kirill A. Shutemov" Cc: Nicholas Piggin , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List This is a multi-part message in MIME format. --------------381B9005558AE185EB668BF2 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 11/07/2017 12:44 PM, Kirill A. Shutemov wrote: > On Tue, Nov 07, 2017 at 12:26:12PM +0100, Florian Weimer wrote: >> On 11/07/2017 12:15 PM, Kirill A. Shutemov wrote: >> >>>> First of all, using addr and MAP_FIXED to develop our heuristic can >>>> never really give unchanged ABI. It's an in-band signal. brk() is a >>>> good example that steadily keeps incrementing address, so depending >>>> on malloc usage and address space randomization, you will get a brk() >>>> that ends exactly at 128T, then the next one will be > >>>> DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. >>> >>> No, it won't. You will hit stack first. >> >> That's not actually true on POWER in some cases. See the process maps I >> posted here: >> >> > > Hm? I see that in all three cases the [stack] is the last mapping. > Do I miss something? Hah, I had not noticed. Occasionally, the order of heap and stack is reversed. This happens in approximately 15% of the runs. See the attached example. Thanks, Florian --------------381B9005558AE185EB668BF2 Content-Type: text/plain; charset=UTF-8; name="maps.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="maps.txt" N2ZmZmFjYzUwMDAwLTdmZmZhY2M5MDAwMCBydy1wIDAwMDAwMDAwIDAwOjAwIDAgCjdmZmZh Y2M5MDAwMC03ZmZmYWNjZjAwMDAgci0tcCAwMDAwMDAwMCBmZDowMCAyNTE2NzkyNSAgICAg ICAgICAgICAgICAgICAvdXNyL2xpYi9sb2NhbGUvZW5fVVMudXRmOC9MQ19DVFlQRQo3ZmZm YWNjZjAwMDAtN2ZmZmFjZDAwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMjUxNjc5MjggICAg ICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfTlVNRVJJQwo3 ZmZmYWNkMDAwMDAtN2ZmZmFjZDEwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5Mjkg ICAgICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfVElNRQo3 ZmZmYWNkMTAwMDAtN2ZmZmFjZTQwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMjUxNjc5MjQg ICAgICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfQ09MTEFU RQo3ZmZmYWNlNDAwMDAtN2ZmZmFjZTUwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5 MjcgICAgICAgICAgICAgICAgICAgL3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfTU9O RVRBUlkKN2ZmZmFjZTUwMDAwLTdmZmZhY2U2MDAwMCByLS1wIDAwMDAwMDAwIGZkOjAwIDI1 MTEgICAgICAgICAgICAgICAgICAgICAgIC91c3IvbGliL2xvY2FsZS9lbl9VUy51dGY4L0xD X01FU1NBR0VTL1NZU19MQ19NRVNTQUdFUwo3ZmZmYWNlNjAwMDAtN2ZmZmFjZTcwMDAwIHIt LXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5NDIgICAgICAgICAgICAgICAgICAgL3Vzci9saWIv bG9jYWxlL2VuX1VTLnV0ZjgvTENfUEFQRVIKN2ZmZmFjZTcwMDAwLTdmZmZhY2U4MDAwMCBy LS1wIDAwMDAwMDAwIGZkOjAwIDI1MTY3OTI3ICAgICAgICAgICAgICAgICAgIC91c3IvbGli L2xvY2FsZS9lbl9VUy51dGY4L0xDX05BTUUKN2ZmZmFjZTgwMDAwLTdmZmZhY2U5MDAwMCBy LS1wIDAwMDAwMDAwIGZkOjAwIDE2Nzk4OTI0ICAgICAgICAgICAgICAgICAgIC91c3IvbGli L2xvY2FsZS9lbl9VUy51dGY4L0xDX0FERFJFU1MKN2ZmZmFjZTkwMDAwLTdmZmZhY2VhMDAw MCByLS1wIDAwMDAwMDAwIGZkOjAwIDE2Nzk4OTI4ICAgICAgICAgICAgICAgICAgIC91c3Iv bGliL2xvY2FsZS9lbl9VUy51dGY4L0xDX1RFTEVQSE9ORQo3ZmZmYWNlYTAwMDAtN2ZmZmFj ZWIwMDAwIHItLXAgMDAwMDAwMDAgZmQ6MDAgMTY3OTg5MjYgICAgICAgICAgICAgICAgICAg L3Vzci9saWIvbG9jYWxlL2VuX1VTLnV0ZjgvTENfTUVBU1VSRU1FTlQKN2ZmZmFjZWIwMDAw LTdmZmZhY2VjMDAwMCByLS1zIDAwMDAwMDAwIGZkOjAwIDgzOTA2NTcgICAgICAgICAgICAg ICAgICAgIC91c3IvbGliNjQvZ2NvbnYvZ2NvbnYtbW9kdWxlcy5jYWNoZQo3ZmZmYWNlYzAw MDAtN2ZmZmFkMGQwMDAwIHIteHAgMDAwMDAwMDAgZmQ6MDAgODM5MDMzNSAgICAgICAgICAg ICAgICAgICAgL3Vzci9saWI2NC9saWJjLTIuMjUuc28KN2ZmZmFkMGQwMDAwLTdmZmZhZDBl MDAwMCAtLS1wIDAwMjEwMDAwIGZkOjAwIDgzOTAzMzUgICAgICAgICAgICAgICAgICAgIC91 c3IvbGliNjQvbGliYy0yLjI1LnNvCjdmZmZhZDBlMDAwMC03ZmZmYWQwZjAwMDAgci0tcCAw MDIxMDAwMCBmZDowMCA4MzkwMzM1ICAgICAgICAgICAgICAgICAgICAvdXNyL2xpYjY0L2xp YmMtMi4yNS5zbwo3ZmZmYWQwZjAwMDAtN2ZmZmFkMTAwMDAwIHJ3LXAgMDAyMjAwMDAgZmQ6 MDAgODM5MDMzNSAgICAgICAgICAgICAgICAgICAgL3Vzci9saWI2NC9saWJjLTIuMjUuc28K N2ZmZmFkMTAwMDAwLTdmZmZhZDExMDAwMCByLS1wIDAwMDAwMDAwIGZkOjAwIDE2Nzk4OTI1 ICAgICAgICAgICAgICAgICAgIC91c3IvbGliL2xvY2FsZS9lbl9VUy51dGY4L0xDX0lERU5U SUZJQ0FUSU9OCjdmZmZhZDExMDAwMC03ZmZmYWQxMjAwMDAgci14cCAwMDAwMDAwMCBmZDow MCA2MzU0MyAgICAgICAgICAgICAgICAgICAgICAvdXNyL2Jpbi9jYXQKN2ZmZmFkMTIwMDAw LTdmZmZhZDEzMDAwMCByLS1wIDAwMDAwMDAwIGZkOjAwIDYzNTQzICAgICAgICAgICAgICAg ICAgICAgIC91c3IvYmluL2NhdAo3ZmZmYWQxMzAwMDAtN2ZmZmFkMTQwMDAwIHJ3LXAgMDAw MTAwMDAgZmQ6MDAgNjM1NDMgICAgICAgICAgICAgICAgICAgICAgL3Vzci9iaW4vY2F0Cjdm ZmZhZDE0MDAwMC03ZmZmYWQxNjAwMDAgci14cCAwMDAwMDAwMCAwMDowMCAwICAgICAgICAg ICAgICAgICAgICAgICAgICBbdmRzb10KN2ZmZmFkMTYwMDAwLTdmZmZhZDFhMDAwMCByLXhw IDAwMDAwMDAwIGZkOjAwIDgzOTAzMjggICAgICAgICAgICAgICAgICAgIC91c3IvbGliNjQv bGQtMi4yNS5zbwo3ZmZmYWQxYTAwMDAtN2ZmZmFkMWIwMDAwIHItLXAgMDAwMzAwMDAgZmQ6 MDAgODM5MDMyOCAgICAgICAgICAgICAgICAgICAgL3Vzci9saWI2NC9sZC0yLjI1LnNvCjdm ZmZhZDFiMDAwMC03ZmZmYWQxYzAwMDAgcnctcCAwMDA0MDAwMCBmZDowMCA4MzkwMzI4ICAg ICAgICAgICAgICAgICAgICAvdXNyL2xpYjY0L2xkLTIuMjUuc28KN2ZmZmMyY2YwMDAwLTdm ZmZjMmQyMDAwMCBydy1wIDAwMDAwMDAwIDAwOjAwIDAgICAgICAgICAgICAgICAgICAgICAg ICAgIFtzdGFja10KN2ZmZmM4YzEwMDAwLTdmZmZjOGM0MDAwMCBydy1wIDAwMDAwMDAwIDAw OjAwIDAgICAgICAgICAgICAgICAgICAgICAgICAgIFtoZWFwXQo= --------------381B9005558AE185EB668BF2-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200]) by kanga.kvack.org (Postfix) with ESMTP id BF58C6B02C2 for ; Tue, 7 Nov 2017 08:16:19 -0500 (EST) Received: by mail-wr0-f200.google.com with SMTP id k100so3697700wrc.9 for ; Tue, 07 Nov 2017 05:16:19 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id d48sor949662eda.20.2017.11.07.05.16.18 for (Google Transport Security); Tue, 07 Nov 2017 05:16:18 -0800 (PST) Date: Tue, 7 Nov 2017 16:16:16 +0300 From: "Kirill A. Shutemov" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107131616.342goolaujjsnjge@node.shutemov.name> References: <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107114422.bgnm5k6w2zqjoazc@node.shutemov.name> <7fc1641b-361c-2ee2-c510-f7c64d173bf8@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7fc1641b-361c-2ee2-c510-f7c64d173bf8@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Florian Weimer Cc: Nicholas Piggin , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, Nov 07, 2017 at 02:05:42PM +0100, Florian Weimer wrote: > On 11/07/2017 12:44 PM, Kirill A. Shutemov wrote: > > On Tue, Nov 07, 2017 at 12:26:12PM +0100, Florian Weimer wrote: > > > On 11/07/2017 12:15 PM, Kirill A. Shutemov wrote: > > > > > > > > First of all, using addr and MAP_FIXED to develop our heuristic can > > > > > never really give unchanged ABI. It's an in-band signal. brk() is a > > > > > good example that steadily keeps incrementing address, so depending > > > > > on malloc usage and address space randomization, you will get a brk() > > > > > that ends exactly at 128T, then the next one will be > > > > > > DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. > > > > > > > > No, it won't. You will hit stack first. > > > > > > That's not actually true on POWER in some cases. See the process maps I > > > posted here: > > > > > > > > > > Hm? I see that in all three cases the [stack] is the last mapping. > > Do I miss something? > > Hah, I had not noticed. Occasionally, the order of heap and stack is > reversed. This happens in approximately 15% of the runs. Heh. I guess ASLR on Power is too fancy :) That's strange layout. It doesn't give that much (relatively speaking) virtual address space for both stack and heap to grow. -- Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f70.google.com (mail-pg0-f70.google.com [74.125.83.70]) by kanga.kvack.org (Postfix) with ESMTP id F11A36B02CA for ; Tue, 7 Nov 2017 08:33:52 -0500 (EST) Received: by mail-pg0-f70.google.com with SMTP id l23so16823465pgc.10 for ; Tue, 07 Nov 2017 05:33:52 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n4sor419216plp.29.2017.11.07.05.33.51 for (Google Transport Security); Tue, 07 Nov 2017 05:33:51 -0800 (PST) Date: Wed, 8 Nov 2017 00:33:32 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171108002448.6799462e@roar.ozlabs.ibm.com> In-Reply-To: <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "Kirill A. Shutemov" Cc: Florian Weimer , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, 7 Nov 2017 15:28:25 +0300 "Kirill A. Shutemov" wrote: > On Tue, Nov 07, 2017 at 10:56:36PM +1100, Nicholas Piggin wrote: > > > No, it won't. You will hit stack first. > > > > I guess so. Florian's bug didn't crash there for some reason, okay > > but I suppose my point about brk is not exactly where the standard > > heap is, but the pattern of allocations. An allocator that uses > > mmap for managing its address space might do the same thing, e.g., > > incrementally expand existing mmaps as necessary. > > With MAP_FIXED? I don't think so. brk() based allocator effectively usees MAP_FIXED. If you know where your addresses are, using MAP_FIXED can be used. But okay let's ignore MAP_FIXED as a corner case. Then what happens with !MAP_FIXED when an allocation ends exactly at 128TB and then next one begins at 128TB? Won't that expand the address space? Should we ignore that corner case too? > > > > > Second, the kernel can never completely solve the problem this way. > > > > How do we know a malloc library will not ask for > 128TB addresses > > > > and pass them to an unknowing application? > > > > > > The idea is that an application can provide hint (mallopt() ?) to malloc > > > implementation that it's ready to full address space. In this case, malloc > > > can use mmap((void *) -1,...) for its allocations and get full address > > > space this way. > > > > Point is, there's nothing stopping an allocator library or runtime > > from asking for mmap anywhere and returning it to userspace. > > Right. Nobody would stop it from doing stupid things. There are many > things that a library may do that application would not be happy about. Indeed. > > > Do > 128TB pointers matter so much that we should add this heuristic > > to prevent breakage, but little enough that we can accept some rare > > cases getting through? Genuine question. > > At the end of the day what matters is if heuristic helps prevent breakage > of existing userspace and doesn't stay in the way of legitimate use of > full address space. > > So far, it looks okay to me. Well that wasn't really the point of my question. Yes of course that is important. But the question is how are these heuristics chosen and evaluated? Why is this a good change to make? We've decided there is some benefit from preventing 128TB pointers, but also not enough that we have to completely prevent them accidentally being returned. Yet it's important enough to make mmap behaviour diverge in about 5 ways around 128TB depending on what combination of address and length and MAP_FIXED you specify, and introducing this new way to use the interface to get an expanded address space? And we're doing this because we don't want to add a prctl or personality or whatever for SAP HANA, because it was decided that would be too complex? > > > > The idea was we shouldn't allow to slip above 47-bits by accidentally. > > > > > > Correctly functioning program would never request addr+len above 47-bit > > > with MAP_FIXED, unless it's ready to handle such addresses. Otherwise the > > > request would simply fail on machine that doesn't support large VA. > > > > > > In contrast, addr+len above 47-bit without MAP_FIXED will not fail on > > > machine that doesn't support large VA, kernel will find another place > > > under 47-bit. And I can imagine a reasonable application that does > > > something like this. > > > > > > So we cannot rely that application is ready to handle large > > > addresses if we see addr+len without MAP_FIXED. > > > > By the same logic, a request for addr > 128TB without MAP_FIXED will > > not fail, therefore we can't rely on that either. > > > > Or an app that links to a library that attempts MAP_FIXED allocation > > of addr + len above 128TB might use high bits of pointer returned by > > that library because those are never satisfied today and the library > > would fall back. > > If you want to point that it's ABI break, yes it is. > > But we allow ABI break as long as nobody notices. I think it's reasonable > to expect that nobody relies on such corner cases. I accept the point, but I do worry mmap syscall is very fundamental, and that it is used in a lot of ways that are difficult to foresee, so there's a lot of room for unintended consequences. Plus the state of the code (that's not to pick on x86, powerpc is worse) is worrying. > > If we would find any piece of software affect by the change we would need > to reconsider. > Problem is that if there was an issue caused by this, it's unlikely to be found until a long time later. By that time, hopefully there aren't too many other apps that are now rely on the behaviour if it has to be changed. I just don't see why that's a risk worth taking at all. I can't see how the cost benefit is there. I have to confess I could have been more involved in discussion, but is any other interface honestly too complex to add? If it is decided to keep these kind of heuristics, can we get just a small but reasonably precise description of each change to the interface and ways for using the new functionality, such that would be suitable for the man page? I couldn't fix powerpc because nothing matches and even Aneesh and you differ on some details (MAP_FIXED behaviour). Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-f198.google.com (mail-qt0-f198.google.com [209.85.216.198]) by kanga.kvack.org (Postfix) with ESMTP id 0CB206B02CB for ; Tue, 7 Nov 2017 08:46:15 -0500 (EST) Received: by mail-qt0-f198.google.com with SMTP id h9so9362147qtc.2 for ; Tue, 07 Nov 2017 05:46:15 -0800 (PST) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com. [148.163.156.1]) by mx.google.com with ESMTPS id o11si1163668qto.480.2017.11.07.05.46.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Nov 2017 05:46:14 -0800 (PST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id vA7DjlkV032618 for ; Tue, 7 Nov 2017 08:46:13 -0500 Received: from e19.ny.us.ibm.com (e19.ny.us.ibm.com [129.33.205.209]) by mx0a-001b2d01.pphosted.com with ESMTP id 2e3dn2sahk-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 07 Nov 2017 08:46:12 -0500 Received: from localhost by e19.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 7 Nov 2017 08:46:10 -0500 Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> From: "Aneesh Kumar K.V" Date: Tue, 7 Nov 2017 19:15:58 +0530 MIME-Version: 1.0 In-Reply-To: <20171108002448.6799462e@roar.ozlabs.ibm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Message-Id: <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin , "Kirill A. Shutemov" Cc: Florian Weimer , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List > > If it is decided to keep these kind of heuristics, can we get just a > small but reasonably precise description of each change to the > interface and ways for using the new functionality, such that would be > suitable for the man page? I couldn't fix powerpc because nothing > matches and even Aneesh and you differ on some details (MAP_FIXED > behaviour). I would consider MAP_FIXED as my mistake. We never discussed this explicitly and I kind of assumed it to behave the same way. ie, we search in lower address space (128TB) if the hint addr is below 128TB. IIUC we agree on the below. 1) MAP_FIXED allow the addr to be used, even if hint addr is below 128TB but hint_addr + len is > 128TB. 2) For everything else we search in < 128TB space if hint addr is below 128TB 3) We don't switch to large address space if hint_addr + len > 128TB. The decision to switch to large address space is primarily based on hint addr Is there any other rule we need to outline? Or is any of the above not correct? -aneesh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f70.google.com (mail-wm0-f70.google.com [74.125.82.70]) by kanga.kvack.org (Postfix) with ESMTP id 403A96B02CF for ; Tue, 7 Nov 2017 09:02:02 -0500 (EST) Received: by mail-wm0-f70.google.com with SMTP id e8so903587wmc.2 for ; Tue, 07 Nov 2017 06:02:02 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id w51sor938646edd.54.2017.11.07.06.02.00 for (Google Transport Security); Tue, 07 Nov 2017 06:02:00 -0800 (PST) Date: Tue, 7 Nov 2017 17:01:58 +0300 From: "Kirill A. Shutemov" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171107140158.iz4b2lchhrt6eobe@node.shutemov.name> References: <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> Sender: owner-linux-mm@kvack.org List-ID: To: "Aneesh Kumar K.V" Cc: Nicholas Piggin , Florian Weimer , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, Nov 07, 2017 at 07:15:58PM +0530, Aneesh Kumar K.V wrote: > > > > > If it is decided to keep these kind of heuristics, can we get just a > > small but reasonably precise description of each change to the > > interface and ways for using the new functionality, such that would be > > suitable for the man page? I couldn't fix powerpc because nothing > > matches and even Aneesh and you differ on some details (MAP_FIXED > > behaviour). > > > I would consider MAP_FIXED as my mistake. We never discussed this explicitly > and I kind of assumed it to behave the same way. ie, we search in lower > address space (128TB) if the hint addr is below 128TB. > > IIUC we agree on the below. > > 1) MAP_FIXED allow the addr to be used, even if hint addr is below 128TB but > hint_addr + len is > 128TB. > > 2) For everything else we search in < 128TB space if hint addr is below > 128TB > > 3) We don't switch to large address space if hint_addr + len > 128TB. The > decision to switch to large address space is primarily based on hint addr > > Is there any other rule we need to outline? Or is any of the above not > correct? That's correct. -- Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl0-f71.google.com (mail-pl0-f71.google.com [209.85.160.71]) by kanga.kvack.org (Postfix) with ESMTP id 415A66B02B4 for ; Tue, 7 Nov 2017 23:56:13 -0500 (EST) Received: by mail-pl0-f71.google.com with SMTP id f6so703611pln.9 for ; Tue, 07 Nov 2017 20:56:13 -0800 (PST) Received: from ozlabs.org (ozlabs.org. [103.22.144.67]) by mx.google.com with ESMTPS id u69si876459pgb.489.2017.11.07.20.56.11 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Nov 2017 20:56:11 -0800 (PST) From: Michael Ellerman Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory In-Reply-To: <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> Date: Wed, 08 Nov 2017 15:56:06 +1100 Message-ID: <87y3nh2wt5.fsf@concordia.ellerman.id.au> MIME-Version: 1.0 Content-Type: text/plain Sender: owner-linux-mm@kvack.org List-ID: To: "Aneesh Kumar K.V" , Nicholas Piggin , "Kirill A. Shutemov" Cc: Florian Weimer , linux-arch@vger.kernel.org, Dave Hansen , Peter Zijlstra , Linus Torvalds , Ingo Molnar , Linux Kernel Mailing List , Andy Lutomirski , linux-mm , Andrew Morton , linuxppc-dev@lists.ozlabs.org, Thomas Gleixner , "Kirill A. Shutemov" "Aneesh Kumar K.V" writes: >> >> If it is decided to keep these kind of heuristics, can we get just a >> small but reasonably precise description of each change to the >> interface and ways for using the new functionality, such that would be >> suitable for the man page? I couldn't fix powerpc because nothing >> matches and even Aneesh and you differ on some details (MAP_FIXED >> behaviour). > > > I would consider MAP_FIXED as my mistake. We never discussed this > explicitly and I kind of assumed it to behave the same way. ie, we > search in lower address space (128TB) if the hint addr is below 128TB. > > IIUC we agree on the below. > > 1) MAP_FIXED allow the addr to be used, even if hint addr is below 128TB > but hint_addr + len is > 128TB. So: mmap(0x7ffffffff000, 0x2000, ..., MAP_FIXED ...) = 0x7ffffffff000 > 2) For everything else we search in < 128TB space if hint addr is below > 128TB mmap((x < 128T), 0x1000, ...) = (y < 128T) ... mmap(0x7ffffffff000, 0x1000, ...) = 0x7ffffffff000 mmap(0x800000000000, 0x1000, ...) = 0x800000000000 ... mmap((x >= 128T), 0x1000, ...) = (y >= 128T) > 3) We don't switch to large address space if hint_addr + len > 128TB. > The decision to switch to large address space is primarily based on hint > addr But does the mmap succeed in that case or not? ie: mmap(0x7ffffffff000, 0x2000, ...) = ? cheers -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f69.google.com (mail-pg0-f69.google.com [74.125.83.69]) by kanga.kvack.org (Postfix) with ESMTP id 7C06E4403E0 for ; Wed, 8 Nov 2017 01:08:27 -0500 (EST) Received: by mail-pg0-f69.google.com with SMTP id r18so1650633pgu.9 for ; Tue, 07 Nov 2017 22:08:27 -0800 (PST) Received: from ozlabs.org (ozlabs.org. [2401:3900:2:1::2]) by mx.google.com with ESMTPS id m17si3129430pfh.214.2017.11.07.22.08.25 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Nov 2017 22:08:25 -0800 (PST) From: Michael Ellerman Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory In-Reply-To: <20171107131616.342goolaujjsnjge@node.shutemov.name> References: <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107114422.bgnm5k6w2zqjoazc@node.shutemov.name> <7fc1641b-361c-2ee2-c510-f7c64d173bf8@redhat.com> <20171107131616.342goolaujjsnjge@node.shutemov.name> Date: Wed, 08 Nov 2017 17:08:20 +1100 Message-ID: <87vail2tgr.fsf@concordia.ellerman.id.au> MIME-Version: 1.0 Content-Type: text/plain Sender: owner-linux-mm@kvack.org List-ID: To: "Kirill A. Shutemov" , Florian Weimer , Kees Cook Cc: linux-arch@vger.kernel.org, Dave Hansen , Peter Zijlstra , Linus Torvalds , Ingo Molnar , Linux Kernel Mailing List , Nicholas Piggin , Andy Lutomirski , linux-mm , "Aneesh Kumar K.V" , Andrew Morton , linuxppc-dev@lists.ozlabs.org, Thomas Gleixner , "Kirill A. Shutemov" "Kirill A. Shutemov" writes: > On Tue, Nov 07, 2017 at 02:05:42PM +0100, Florian Weimer wrote: >> On 11/07/2017 12:44 PM, Kirill A. Shutemov wrote: >> > On Tue, Nov 07, 2017 at 12:26:12PM +0100, Florian Weimer wrote: >> > > On 11/07/2017 12:15 PM, Kirill A. Shutemov wrote: >> > > >> > > > > First of all, using addr and MAP_FIXED to develop our heuristic can >> > > > > never really give unchanged ABI. It's an in-band signal. brk() is a >> > > > > good example that steadily keeps incrementing address, so depending >> > > > > on malloc usage and address space randomization, you will get a brk() >> > > > > that ends exactly at 128T, then the next one will be > >> > > > > DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. >> > > > >> > > > No, it won't. You will hit stack first. >> > > >> > > That's not actually true on POWER in some cases. See the process maps I >> > > posted here: >> > > >> > > >> > >> > Hm? I see that in all three cases the [stack] is the last mapping. >> > Do I miss something? >> >> Hah, I had not noticed. Occasionally, the order of heap and stack is >> reversed. This happens in approximately 15% of the runs. > > Heh. I guess ASLR on Power is too fancy :) Fancy implies we're doing it on purpose :P > That's strange layout. It doesn't give that much (relatively speaking) > virtual address space for both stack and heap to grow. I'm pretty sure it only happens when you're running an ELF interpreter directly, because of Kees patch which changed the logic to load ELF interpreters in the mmap region, vs PIE binaries which go to ELF_ET_DYN_BASE. (eab09532d400 ("binfmt_elf: use ELF_ET_DYN_BASE only for PIE")) It only happens with ASLR enabled. Presumably it's because our brk_rnd() is overly aggressive in this case, it randomises up to 1GB, and the heap jumps over the stack. cheers -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f72.google.com (mail-oi0-f72.google.com [209.85.218.72]) by kanga.kvack.org (Postfix) with ESMTP id A16944403E0 for ; Wed, 8 Nov 2017 01:18:24 -0500 (EST) Received: by mail-oi0-f72.google.com with SMTP id h6so1365970oia.17 for ; Tue, 07 Nov 2017 22:18:24 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id o70si1443341oik.207.2017.11.07.22.18.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Nov 2017 22:18:23 -0800 (PST) Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory References: <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107114422.bgnm5k6w2zqjoazc@node.shutemov.name> <7fc1641b-361c-2ee2-c510-f7c64d173bf8@redhat.com> <20171107131616.342goolaujjsnjge@node.shutemov.name> <87vail2tgr.fsf@concordia.ellerman.id.au> From: Florian Weimer Message-ID: <9d5c86e9-d011-76b4-6357-b6009a201cdb@redhat.com> Date: Wed, 8 Nov 2017 07:18:17 +0100 MIME-Version: 1.0 In-Reply-To: <87vail2tgr.fsf@concordia.ellerman.id.au> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Michael Ellerman , "Kirill A. Shutemov" , Kees Cook Cc: linux-arch@vger.kernel.org, Dave Hansen , Peter Zijlstra , Linus Torvalds , Ingo Molnar , Linux Kernel Mailing List , Nicholas Piggin , Andy Lutomirski , linux-mm , "Aneesh Kumar K.V" , Andrew Morton , linuxppc-dev@lists.ozlabs.org, Thomas Gleixner , "Kirill A. Shutemov" On 11/08/2017 07:08 AM, Michael Ellerman wrote: > "Kirill A. Shutemov" writes: > >> On Tue, Nov 07, 2017 at 02:05:42PM +0100, Florian Weimer wrote: >>> On 11/07/2017 12:44 PM, Kirill A. Shutemov wrote: >>>> On Tue, Nov 07, 2017 at 12:26:12PM +0100, Florian Weimer wrote: >>>>> On 11/07/2017 12:15 PM, Kirill A. Shutemov wrote: >>>>> >>>>>>> First of all, using addr and MAP_FIXED to develop our heuristic can >>>>>>> never really give unchanged ABI. It's an in-band signal. brk() is a >>>>>>> good example that steadily keeps incrementing address, so depending >>>>>>> on malloc usage and address space randomization, you will get a brk() >>>>>>> that ends exactly at 128T, then the next one will be > >>>>>>> DEFAULT_MAP_WINDOW, and it will switch you to 56 bit address space. >>>>>> >>>>>> No, it won't. You will hit stack first. >>>>> >>>>> That's not actually true on POWER in some cases. See the process maps I >>>>> posted here: >>>>> >>>>> >>>> >>>> Hm? I see that in all three cases the [stack] is the last mapping. >>>> Do I miss something? >>> >>> Hah, I had not noticed. Occasionally, the order of heap and stack is >>> reversed. This happens in approximately 15% of the runs. >> >> Heh. I guess ASLR on Power is too fancy :) > > Fancy implies we're doing it on purpose :P > >> That's strange layout. It doesn't give that much (relatively speaking) >> virtual address space for both stack and heap to grow. > > I'm pretty sure it only happens when you're running an ELF interpreter > directly, because of Kees patch which changed the logic to load ELF > interpreters in the mmap region, vs PIE binaries which go to > ELF_ET_DYN_BASE. (eab09532d400 ("binfmt_elf: use ELF_ET_DYN_BASE only > for PIE")) From that commit: + * There are effectively two types of ET_DYN + * binaries: programs (i.e. PIE: ET_DYN with INTERP) + * and loaders (ET_DYN without INTERP, since they + * _are_ the ELF interpreter). The loaders must Note that the comment is a bit misleading: statically linked PIE binaries are ET_DYN without INTERP, too. So any oddity which is observable today with an explicitly ld.so invocation only will gain more relevance once we get static PIE support in user space because it will then affect regular applications, too. (Well, statically linked ones.) In this sense, process layouts which cause premature brk failure or insufficient stack allocations are real bugs. Thanks, Florian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199]) by kanga.kvack.org (Postfix) with ESMTP id ECEA44403E0 for ; Wed, 8 Nov 2017 03:30:41 -0500 (EST) Received: by mail-wr0-f199.google.com with SMTP id w95so976485wrc.20 for ; Wed, 08 Nov 2017 00:30:41 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id o30sor2205374eda.56.2017.11.08.00.30.40 for (Google Transport Security); Wed, 08 Nov 2017 00:30:40 -0800 (PST) Date: Wed, 8 Nov 2017 11:30:38 +0300 From: "Kirill A. Shutemov" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171108083038.djj3qdhhnrazkk7b@node.shutemov.name> References: <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> <87y3nh2wt5.fsf@concordia.ellerman.id.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87y3nh2wt5.fsf@concordia.ellerman.id.au> Sender: owner-linux-mm@kvack.org List-ID: To: Michael Ellerman Cc: "Aneesh Kumar K.V" , Nicholas Piggin , Florian Weimer , linux-arch@vger.kernel.org, Dave Hansen , Peter Zijlstra , Linus Torvalds , Ingo Molnar , Linux Kernel Mailing List , Andy Lutomirski , linux-mm , Andrew Morton , linuxppc-dev@lists.ozlabs.org, Thomas Gleixner , "Kirill A. Shutemov" On Wed, Nov 08, 2017 at 03:56:06PM +1100, Michael Ellerman wrote: > "Aneesh Kumar K.V" writes: > > >> > >> If it is decided to keep these kind of heuristics, can we get just a > >> small but reasonably precise description of each change to the > >> interface and ways for using the new functionality, such that would be > >> suitable for the man page? I couldn't fix powerpc because nothing > >> matches and even Aneesh and you differ on some details (MAP_FIXED > >> behaviour). > > > > > > I would consider MAP_FIXED as my mistake. We never discussed this > > explicitly and I kind of assumed it to behave the same way. ie, we > > search in lower address space (128TB) if the hint addr is below 128TB. > > > > IIUC we agree on the below. > > > > 1) MAP_FIXED allow the addr to be used, even if hint addr is below 128TB > > but hint_addr + len is > 128TB. > > So: > mmap(0x7ffffffff000, 0x2000, ..., MAP_FIXED ...) = 0x7ffffffff000 > > > 2) For everything else we search in < 128TB space if hint addr is below > > 128TB > > mmap((x < 128T), 0x1000, ...) = (y < 128T) > ... > mmap(0x7ffffffff000, 0x1000, ...) = 0x7ffffffff000 > mmap(0x800000000000, 0x1000, ...) = 0x800000000000 > ... > mmap((x >= 128T), 0x1000, ...) = (y >= 128T) > > > 3) We don't switch to large address space if hint_addr + len > 128TB. > > The decision to switch to large address space is primarily based on hint > > addr > > But does the mmap succeed in that case or not? > > ie: mmap(0x7ffffffff000, 0x2000, ...) = ? It does, but resulting address doesn't match the hint. It's somewhere below 47-bit border. -- Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f200.google.com (mail-pf0-f200.google.com [209.85.192.200]) by kanga.kvack.org (Postfix) with ESMTP id 4210F440D03 for ; Thu, 9 Nov 2017 12:15:43 -0500 (EST) Received: by mail-pf0-f200.google.com with SMTP id y128so5168932pfg.5 for ; Thu, 09 Nov 2017 09:15:43 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id m74sor1721044pga.66.2017.11.09.09.15.42 for (Google Transport Security); Thu, 09 Nov 2017 09:15:42 -0800 (PST) Date: Fri, 10 Nov 2017 04:15:26 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171110041526.6137bc9a@roar.ozlabs.ibm.com> In-Reply-To: <20171107140158.iz4b2lchhrt6eobe@node.shutemov.name> References: <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> <20171107140158.iz4b2lchhrt6eobe@node.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "Kirill A. Shutemov" Cc: "Aneesh Kumar K.V" , Florian Weimer , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Tue, 7 Nov 2017 17:01:58 +0300 "Kirill A. Shutemov" wrote: > On Tue, Nov 07, 2017 at 07:15:58PM +0530, Aneesh Kumar K.V wrote: > > > > > > > > If it is decided to keep these kind of heuristics, can we get just a > > > small but reasonably precise description of each change to the > > > interface and ways for using the new functionality, such that would be > > > suitable for the man page? I couldn't fix powerpc because nothing > > > matches and even Aneesh and you differ on some details (MAP_FIXED > > > behaviour). > > > > > > I would consider MAP_FIXED as my mistake. We never discussed this explicitly > > and I kind of assumed it to behave the same way. ie, we search in lower > > address space (128TB) if the hint addr is below 128TB. > > > > IIUC we agree on the below. > > > > 1) MAP_FIXED allow the addr to be used, even if hint addr is below 128TB but > > hint_addr + len is > 128TB. > > > > 2) For everything else we search in < 128TB space if hint addr is below > > 128TB > > > > 3) We don't switch to large address space if hint_addr + len > 128TB. The > > decision to switch to large address space is primarily based on hint addr > > > > Is there any other rule we need to outline? Or is any of the above not > > correct? > > That's correct. > Thanks guys, I'll send out some powerpc patches to match -- it deviates in its MAP_FIXED handling (treats it the same as !MAP_FIXED). So these semantics are what we're going with? Anything that does mmap() is guaranteed of getting a 47-bit pointer and it can use the top 17 bits for itself? Is intended to be cross-platform or just x86 and power specific? Also, this may follow from deduction from 1-3, but for explicit specification in man page: 4) To get an unspecified allocation with the largest possible address range, we pass in -1 for mmap hint. Are we allowing 8 bits bits of unused address in this case, or must the app not assume anything about number of bits used? Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f69.google.com (mail-pg0-f69.google.com [74.125.83.69]) by kanga.kvack.org (Postfix) with ESMTP id 1A5D0440D03 for ; Thu, 9 Nov 2017 14:44:28 -0500 (EST) Received: by mail-pg0-f69.google.com with SMTP id 192so75194pgd.18 for ; Thu, 09 Nov 2017 11:44:28 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org. [65.50.211.133]) by mx.google.com with ESMTPS id a28si6968291pgd.464.2017.11.09.11.44.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Nov 2017 11:44:26 -0800 (PST) Date: Thu, 9 Nov 2017 11:44:21 -0800 From: Matthew Wilcox Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171109194421.GA12789@bombadil.infradead.org> References: <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> <20171107140158.iz4b2lchhrt6eobe@node.shutemov.name> <20171110041526.6137bc9a@roar.ozlabs.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171110041526.6137bc9a@roar.ozlabs.ibm.com> Sender: owner-linux-mm@kvack.org List-ID: To: Nicholas Piggin Cc: "Kirill A. Shutemov" , "Aneesh Kumar K.V" , Florian Weimer , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Fri, Nov 10, 2017 at 04:15:26AM +1100, Nicholas Piggin wrote: > So these semantics are what we're going with? Anything that does mmap() is > guaranteed of getting a 47-bit pointer and it can use the top 17 bits for > itself? Is intended to be cross-platform or just x86 and power specific? It is x86 and powerpc specific. The arm64 people have apparently stumbled across apps that expect to be able to use bit 48 for their own purposes. And their address space is 48 bit by default. Oops. > Also, this may follow from deduction from 1-3, but for explicit > specification in man page: > > 4) To get an unspecified allocation with the largest possible address range, > we pass in -1 for mmap hint. > > Are we allowing 8 bits bits of unused address in this case, or must the > app not assume anything about number of bits used? Maybe document it as: "If the app wants to use the top N bits of addresses for its own purposes, pass in (~0UL >> N) as the mmap hint." ? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f71.google.com (mail-pg0-f71.google.com [74.125.83.71]) by kanga.kvack.org (Postfix) with ESMTP id E458E440D03 for ; Thu, 9 Nov 2017 20:26:44 -0500 (EST) Received: by mail-pg0-f71.google.com with SMTP id 4so5852252pge.8 for ; Thu, 09 Nov 2017 17:26:44 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y24sor663584pll.125.2017.11.09.17.26.43 for (Google Transport Security); Thu, 09 Nov 2017 17:26:43 -0800 (PST) Date: Fri, 10 Nov 2017 12:26:25 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171110122625.4ea3c897@roar.ozlabs.ibm.com> In-Reply-To: <20171109194421.GA12789@bombadil.infradead.org> References: <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> <20171107140158.iz4b2lchhrt6eobe@node.shutemov.name> <20171110041526.6137bc9a@roar.ozlabs.ibm.com> <20171109194421.GA12789@bombadil.infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Matthew Wilcox Cc: "Kirill A. Shutemov" , "Aneesh Kumar K.V" , Florian Weimer , "Kirill A. Shutemov" , linuxppc-dev@lists.ozlabs.org, linux-mm , Andrew Morton , Andy Lutomirski , Dave Hansen , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , linux-arch@vger.kernel.org, Ingo Molnar , Linux Kernel Mailing List On Thu, 9 Nov 2017 11:44:21 -0800 Matthew Wilcox wrote: > On Fri, Nov 10, 2017 at 04:15:26AM +1100, Nicholas Piggin wrote: > > So these semantics are what we're going with? Anything that does mmap() is > > guaranteed of getting a 47-bit pointer and it can use the top 17 bits for > > itself? Is intended to be cross-platform or just x86 and power specific? > > It is x86 and powerpc specific. The arm64 people have apparently stumbled > across apps that expect to be able to use bit 48 for their own purposes. > And their address space is 48 bit by default. Oops. Okay, so it's something we should make into an "official" API? > > > Also, this may follow from deduction from 1-3, but for explicit > > specification in man page: > > > > 4) To get an unspecified allocation with the largest possible address range, > > we pass in -1 for mmap hint. > > > > Are we allowing 8 bits bits of unused address in this case, or must the > > app not assume anything about number of bits used? > > Maybe document it as: "If the app wants to use the top N bits of addresses > for its own purposes, pass in (~0UL >> N) as the mmap hint." ? Well we don't have code for that yet, but the problem would also be that it succeeds, and actually it probably goes over the limit. So you would have to map a dummy page there so subsequent hints to fail and fall back. Not sure... it would be nice to be able to specify number of bits, but I think this gets a bit hairy. -1 to use all bits seems a bit easier. Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f200.google.com (mail-pf0-f200.google.com [209.85.192.200]) by kanga.kvack.org (Postfix) with ESMTP id 12AFB28027D for ; Sat, 11 Nov 2017 05:31:07 -0500 (EST) Received: by mail-pf0-f200.google.com with SMTP id r6so9888614pfj.14 for ; Sat, 11 Nov 2017 02:31:07 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id p1sor3032537pfi.4.2017.11.11.02.31.05 for (Google Transport Security); Sat, 11 Nov 2017 02:31:05 -0800 (PST) Date: Sat, 11 Nov 2017 21:30:50 +1100 From: Nicholas Piggin Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Message-ID: <20171111213050.34a4f585@roar.ozlabs.ibm.com> In-Reply-To: <063D6719AE5E284EB5DD2968C1650D6DD00B84EF@AcuExch.aculab.com> References: <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> <20171107140158.iz4b2lchhrt6eobe@node.shutemov.name> <20171110041526.6137bc9a@roar.ozlabs.ibm.com> <20171109194421.GA12789@bombadil.infradead.org> <063D6719AE5E284EB5DD2968C1650D6DD00B84EF@AcuExch.aculab.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: David Laight Cc: 'Matthew Wilcox' , Florian Weimer , "linux-arch@vger.kernel.org" , Dave Hansen , Peter Zijlstra , Linus Torvalds , Ingo Molnar , Linux Kernel Mailing List , Andy Lutomirski , linux-mm , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , Andrew Morton , "linuxppc-dev@lists.ozlabs.org" , Thomas Gleixner , "Kirill A. Shutemov" On Fri, 10 Nov 2017 12:08:35 +0000 David Laight wrote: > From: Matthew Wilcox > > Sent: 09 November 2017 19:44 > > > > On Fri, Nov 10, 2017 at 04:15:26AM +1100, Nicholas Piggin wrote: > > > So these semantics are what we're going with? Anything that does mmap() is > > > guaranteed of getting a 47-bit pointer and it can use the top 17 bits for > > > itself? Is intended to be cross-platform or just x86 and power specific? > > > > It is x86 and powerpc specific. The arm64 people have apparently stumbled > > across apps that expect to be able to use bit 48 for their own purposes. > > And their address space is 48 bit by default. Oops. > > (Do you mean 49bit?) I think he meant bit 47, which makes sense because they were probably ported from x86-64 with 47 bit address. That seems to be why x86-64 5-level and powerpc decided to limit to a 47 bit address space by default. > > Aren't such apps just doomed to be broken? Well they're not portable but they are not broken if virtual address is limited. > > ISTR there is something on (IIRC) sparc64 that does a 'match' > on the high address bits to make it much harder to overrun > one area into another. I'm not sure about that but I think the problem would be the app masking out bits from the pointer for its own use before ever dereferencing it. Thanks, Nick -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Aneesh Kumar K.V" Subject: Re: POWER: Unexpected fault when writing to brk-allocated memory Date: Mon, 6 Nov 2017 14:00:01 +0530 Message-ID: References: <20171105231850.5e313e46@roar.ozlabs.ibm.com> <871slcszfl.fsf@linux.vnet.ibm.com> <20171106174707.19f6c495@roar.ozlabs.ibm.com> <24b93038-76f7-33df-d02e-facb0ce61cd2@redhat.com> <20171106192524.12ea3187@roar.ozlabs.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20171106192524.12ea3187@roar.ozlabs.ibm.com> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+glppe-linuxppc-embedded-2=m.gmane.org@lists.ozlabs.org Sender: "Linuxppc-dev" To: Nicholas Piggin , Florian Weimer Cc: linux-mm , linuxppc-dev@lists.ozlabs.org, "Kirill A. Shutemov" List-Id: linux-mm.kvack.org On 11/06/2017 01:55 PM, Nicholas Piggin wrote: > On Mon, 6 Nov 2017 09:11:37 +0100 > Florian Weimer wrote: > >> On 11/06/2017 07:47 AM, Nicholas Piggin wrote: >>> "You get < 128TB unless explicitly requested." >>> >>> Simple, reasonable, obvious rule. Avoids breaking apps that store >>> some bits in the top of pointers (provided that memory allocator >>> userspace libraries also do the right thing). >> >> So brk would simplify fail instead of crossing the 128 TiB threshold? > > Yes, that was the intention and that's what x86 seems to do. > >> >> glibc malloc should cope with that and switch to malloc, but this code >> path is obviously less well-tested than the regular way. > > Switch to mmap() I guess you meant? > > powerpc has a couple of bugs in corner cases, so those should be fixed > according to intended policy for stable kernels I think. > > But I question the policy. Just seems like an ugly and ineffective wart. > Exactly for such cases as this -- behaviour would change from run to run > depending on your address space randomization for example! In case your > brk happens to land nicely on 128TB then the next one would succeed. Why ? It should not change between run to run. We limit the free area search range based on hint address. So we should get consistent results across run. even if we changed the context.addr_limit. -aneesh From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Laight Subject: RE: POWER: Unexpected fault when writing to brk-allocated memory Date: Fri, 10 Nov 2017 12:08:35 +0000 Message-ID: <063D6719AE5E284EB5DD2968C1650D6DD00B84EF@AcuExch.aculab.com> References: <546d4155-5b7c-6dba-b642-29c103e336bc@redhat.com> <20171107160705.059e0c2b@roar.ozlabs.ibm.com> <20171107111543.ep57evfxxbwwlhdh@node.shutemov.name> <20171107222228.0c8a50ff@roar.ozlabs.ibm.com> <20171107122825.posamr2dmzlzvs2p@node.shutemov.name> <20171108002448.6799462e@roar.ozlabs.ibm.com> <2ce0a91c-985c-aad8-abfa-e91bc088bb3e@linux.vnet.ibm.com> <20171107140158.iz4b2lchhrt6eobe@node.shutemov.name> <20171110041526.6137bc9a@roar.ozlabs.ibm.com> <20171109194421.GA12789@bombadil.infradead.org> Mime-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20171109194421.GA12789@bombadil.infradead.org> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+glppe-linuxppc-embedded-2=m.gmane.org@lists.ozlabs.org Sender: "Linuxppc-dev" To: 'Matthew Wilcox' , Nicholas Piggin Cc: Florian Weimer , "linux-arch@vger.kernel.org" , linux-mm , Peter Zijlstra , "linuxppc-dev@lists.ozlabs.org" , Linux Kernel Mailing List , Andy Lutomirski , Dave Hansen , Thomas Gleixner , "Aneesh Kumar K.V" , "Kirill A. Shutemov" , Andrew Morton , Linus Torvalds , Ingo Molnar , "Kirill A. Shutemov" List-Id: linux-mm.kvack.org From: Matthew Wilcox > Sent: 09 November 2017 19:44 >=20 > On Fri, Nov 10, 2017 at 04:15:26AM +1100, Nicholas Piggin wrote: > > So these semantics are what we're going with? Anything that does mmap()= is > > guaranteed of getting a 47-bit pointer and it can use the top 17 bits f= or > > itself? Is intended to be cross-platform or just x86 and power specific= ? >=20 > It is x86 and powerpc specific. The arm64 people have apparently stumble= d > across apps that expect to be able to use bit 48 for their own purposes. > And their address space is 48 bit by default. Oops. (Do you mean 49bit?) Aren't such apps just doomed to be broken? ISTR there is something on (IIRC) sparc64 that does a 'match' on the high address bits to make it much harder to overrun one area into another. David