From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1m2vAK-0002Nc-4c for mharc-grub-devel@gnu.org; Mon, 12 Jul 2021 08:36:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m2vAH-0002JK-Rx for grub-devel@gnu.org; Mon, 12 Jul 2021 08:36:01 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:20632) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m2vAE-0003Hz-2a for grub-devel@gnu.org; Mon, 12 Jul 2021 08:36:00 -0400 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16CCZM6f133413; Mon, 12 Jul 2021 08:35:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=u+fEiQ6voOdGcGe3cdW+debOZYF4LiFq9quKDbJ5WS8=; b=XlJWFtLvKmj1kcvUCh+kFqlUG/GMCtfO6XEqFZCgdJNCocGB6n3onjJi0a/VKVcnhxm/ /QnHl0H3F46E0qF3C8+rkiCrwvZ7ZgHDLBAxygT/o0KY9lKldKVIh/2eNKDw6efLsl7Y akSw+/TikbDVZzHLkDQNY/J7F0XnmBk0CQloL1AYLnvW6yDKQ5xmaZHDlX/wRrIzKJA9 s7dkswvBwzTDPjHNR6ZokVweEZJ4Y1UDtcQgQRVQ4tv0C5RmiFMva0dakc2EjXrzcAIF 6mVYw9lTO4bj/vB6S9ERWpLNh16iNc1JYZsC2lGXT/OqtqsZdLpLH6Bsdi5eZ4VeTpla Sg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 39qrmbrskc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 12 Jul 2021 08:35:55 -0400 Received: from m0098393.ppops.net (m0098393.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 16CCZsuE137929; Mon, 12 Jul 2021 08:35:54 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 39qrmbrsjg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 12 Jul 2021 08:35:54 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 16CCRVca006765; Mon, 12 Jul 2021 12:35:53 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma05wdc.us.ibm.com with ESMTP id 39q36b2rpm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 12 Jul 2021 12:35:53 +0000 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 16CCZroo53149982 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 12 Jul 2021 12:35:53 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 12D3AAE062; Mon, 12 Jul 2021 12:35:53 +0000 (GMT) Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 00040AE06A; Mon, 12 Jul 2021 12:35:52 +0000 (GMT) Received: from [9.47.158.152] (unknown [9.47.158.152]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP; Mon, 12 Jul 2021 12:35:52 +0000 (GMT) Subject: Re: [PATCH v2 02/22] ieee1275: claim more memory To: The development of GNU GRUB , Daniel Axtens Cc: rashmica.g@gmail.com, alastair@d-silva.org, nayna@linux.ibm.com References: <20210630084031.2663622-1-dja@axtens.net> <20210630084031.2663622-3-dja@axtens.net> From: Stefan Berger Message-ID: <05967f2f-2d68-bc87-9b63-08fa58a0740f@linux.ibm.com> Date: Mon, 12 Jul 2021 08:35:52 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210630084031.2663622-3-dja@axtens.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: qwxkyJhnE3wnL_vCIH422lR_l603zHNp X-Proofpoint-GUID: J2QAPQF04pHZsHvi6Fk9e4T2ZJtE3zvn X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-07-12_07:2021-07-12, 2021-07-12 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 phishscore=0 mlxscore=0 lowpriorityscore=0 adultscore=0 malwarescore=0 suspectscore=0 impostorscore=0 mlxlogscore=999 priorityscore=1501 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107120099 Received-SPF: pass client-ip=148.163.156.1; envelope-from=stefanb@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-1.479, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2021 12:36:02 -0000 On 6/30/21 4:40 AM, Daniel Axtens wrote: > On powerpc-ieee1275, we are running out of memory trying to verify > anything. This is because: > > - we have to load an entire file into memory to verify it. This is > extremely difficult to change with appended signatures. > - We only have 32MB of heap. > - Distro kernels are now often around 30MB. > > So we want to claim more memory from OpenFirmware for our heap. > > There are some complications: > > - The grub mm code isn't the only thing that will make claims on > memory from OpenFirmware: > > * PFW/SLOF will have claimed some for their own use. > > * The ieee1275 loader will try to find other bits of memory that we > haven't claimed to place the kernel and initrd when we go to boot. > > * Once we load Linux, it will also try to claim memory. It claims > memory without any reference to /memory/available, it just starts > at min(top of RMO, 768MB) and works down. So we need to avoid this > area. See arch/powerpc/kernel/prom_init.c as of v5.11. > > - The smallest amount of memory a ppc64 KVM guest can have is 256MB. > It doesn't work with distro kernels but can work with custom kernels. > We should maintain support for that. (ppc32 can boot with even less, > and we shouldn't break that either.) > > - Even if a VM has more memory, the memory OpenFirmware makes available > as Real Memory Area can be restricted. A freshly created LPAR on a > PowerVM machine is likely to have only 256MB available to OpenFirmware > even if it has many gigabytes of memory allocated. > > EFI systems will attempt to allocate 1/4th of the available memory, > clamped to between 1M and 1600M. That seems like a good sort of > approach, we just need to figure out if 1/4 is the right fraction > for us. > > We don't know in advance how big the kernel and initrd are going to be, > which makes figuring out how much memory we can take a bit tricky. > > To figure out how much memory we should leave unused, I looked at: > > - an Ubuntu 20.04.1 ppc64le pseries KVM guest: > vmlinux: ~30MB > initrd: ~50MB > > - a RHEL8.2 ppc64le pseries KVM guest: > vmlinux: ~30MB > initrd: ~30MB > > Ubuntu VMs struggle to boot with just 256MB under SLOF. > RHEL likewise has a higher minimum supported memory figure. > So lets first consider a distro kernel and 512MB of addressible memory. > (This is the default case for anything booting under PFW.) Say we lose > 131MB to PFW (based on some tests). This leaves us 381MB. 1/4 of 381MB > is ~95MB. That should be enough to verify a 30MB vmlinux and should > leave plenty of space to load Linux and the initrd. > > If we consider 256MB of RMA under PFW, we have just 125MB remaining. 1/4 > of that is a smidge under 32MB, which gives us very poor odds of verifying > a distro-sized kernel. However, if we need 80MB just to put the kernel > and initrd in memory, we can't claim any more than 45MB anyway. So 1/4 > will do. We'll come back to this later. > > grub is always built as a 32-bit binary, even if it's loading a ppc64 > kernel. So we can't address memory beyond 4GB. This gives a natural cap > of 1GB for powerpc-ieee1275. > > Also apply this 1/4 approach to i386-ieee1275, but keep the 32MB cap. > > make check still works for both i386 and powerpc and I've booted > powerpc grub with this change under SLOF and PFW. > > Signed-off-by: Daniel Axtens Tested-by: Stefan Berger > --- > docs/grub-dev.texi | 6 ++- > grub-core/kern/ieee1275/init.c | 81 +++++++++++++++++++++++++++------- > 2 files changed, 69 insertions(+), 18 deletions(-) > > diff --git a/docs/grub-dev.texi b/docs/grub-dev.texi > index 6c629a23e2dc..c11f1ac46de2 100644 > --- a/docs/grub-dev.texi > +++ b/docs/grub-dev.texi > @@ -1047,7 +1047,9 @@ space is limited to 4GiB. GRUB allocates pages from EFI for its heap, at most > 1.6 GiB. > > On i386-ieee1275 and powerpc-ieee1275 GRUB uses same stack as IEEE1275. > -It allocates at most 32MiB for its heap. > + > +On i386-ieee1275, GRUB allocates at most 32MiB for its heap. On > +powerpc-ieee1275, GRUB allocates up to 1GiB. > > On sparc64-ieee1275 stack is 256KiB and heap is 2MiB. > > @@ -1075,7 +1077,7 @@ In short: > @item i386-qemu @tab 60 KiB @tab < 4 GiB > @item *-efi @tab ? @tab < 1.6 GiB > @item i386-ieee1275 @tab ? @tab < 32 MiB > -@item powerpc-ieee1275 @tab ? @tab < 32 MiB > +@item powerpc-ieee1275 @tab ? @tab < 1 GiB > @item sparc64-ieee1275 @tab 256KiB @tab 2 MiB > @item arm-uboot @tab 256KiB @tab 2 MiB > @item mips(el)-qemu_mips @tab 2MiB @tab 253 MiB > diff --git a/grub-core/kern/ieee1275/init.c b/grub-core/kern/ieee1275/init.c > index c5d091689f29..4162b5949df4 100644 > --- a/grub-core/kern/ieee1275/init.c > +++ b/grub-core/kern/ieee1275/init.c > @@ -45,11 +45,12 @@ > #include > #endif > > -/* The maximum heap size we're going to claim */ > +/* The maximum heap size we're going to claim. Not used by sparc. > + We allocate 1/4 of the available memory under 4G, up to this limit. */ > #ifdef __i386__ > #define HEAP_MAX_SIZE (unsigned long) (64 * 1024 * 1024) > -#else > -#define HEAP_MAX_SIZE (unsigned long) (32 * 1024 * 1024) > +#else // __powerpc__ > +#define HEAP_MAX_SIZE (unsigned long) (1 * 1024 * 1024 * 1024) > #endif > > extern char _start[]; > @@ -145,16 +146,45 @@ grub_claim_heap (void) > + GRUB_KERNEL_MACHINE_STACK_SIZE), 0x200000); > } > #else > -/* Helper for grub_claim_heap. */ > +/* Helper for grub_claim_heap on powerpc. */ > +static int > +heap_size (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > + void *data) > +{ > + grub_uint32_t total = *(grub_uint32_t *)data; > + > + if (type != GRUB_MEMORY_AVAILABLE) > + return 0; > + > + /* Do not consider memory beyond 4GB */ > + if (addr > 0xffffffffUL) > + return 0; > + > + if (addr + len > 0xffffffffUL) > + len = 0xffffffffUL - addr; > + > + total += len; > + *(grub_uint32_t *)data = total; > + > + return 0; > +} > + > static int > heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > void *data) > { > - unsigned long *total = data; > + grub_uint32_t total = *(grub_uint32_t *)data; > > if (type != GRUB_MEMORY_AVAILABLE) > return 0; > > + /* Do not consider memory beyond 4GB */ > + if (addr > 0xffffffffUL) > + return 0; > + > + if (addr + len > 0xffffffffUL) > + len = 0xffffffffUL - addr; > + > if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_NO_PRE1_5M_CLAIM)) > { > if (addr + len <= 0x180000) > @@ -168,10 +198,6 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > } > len -= 1; /* Required for some firmware. */ > > - /* Never exceed HEAP_MAX_SIZE */ > - if (*total + len > HEAP_MAX_SIZE) > - len = HEAP_MAX_SIZE - *total; > - > /* In theory, firmware should already prevent this from happening by not > listing our own image in /memory/available. The check below is intended > as a safeguard in case that doesn't happen. However, it doesn't protect > @@ -183,6 +209,18 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > len = 0; > } > > + /* If this block contains 0x30000000 (768MB), do not claim below that. > + Linux likes to claim memory at min(RMO top, 768MB) and works down > + without reference to /memory/available. */ > + if ((addr < 0x30000000) && ((addr + len) > 0x30000000)) > + { > + len = len - (0x30000000 - addr); > + addr = 0x30000000; > + } > + > + if (len > total) > + len = total; > + > if (len) > { > grub_err_t err; > @@ -191,10 +229,12 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > if (err) > return err; > grub_mm_init_region ((void *) (grub_addr_t) addr, len); > + total -= len; > } > > - *total += len; > - if (*total >= HEAP_MAX_SIZE) > + *(grub_uint32_t *)data = total; > + > + if (total == 0) > return 1; > > return 0; > @@ -203,13 +243,22 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > static void > grub_claim_heap (void) > { > - unsigned long total = 0; > + grub_uint32_t total = 0; > > if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_FORCE_CLAIM)) > - heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN, > - 1, &total); > - else > - grub_machine_mmap_iterate (heap_init, &total); > + { > + heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN, > + 1, &total); > + return; > + } > + > + grub_machine_mmap_iterate (heap_size, &total); > + > + total = total / 4; > + if (total > HEAP_MAX_SIZE) > + total = HEAP_MAX_SIZE; > + > + grub_machine_mmap_iterate (heap_init, &total); > } > #endif >