From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 470D9C07E9D for ; Sat, 24 Sep 2022 09:36:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DD4D6B009E; Sat, 24 Sep 2022 05:36:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78CE68E0005; Sat, 24 Sep 2022 05:36:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 654C78E0001; Sat, 24 Sep 2022 05:36:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5282D6B009E for ; Sat, 24 Sep 2022 05:36:37 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1915EC0D8E for ; Sat, 24 Sep 2022 09:36:37 +0000 (UTC) X-FDA: 79946474034.08.B79551A Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf24.hostedemail.com (Postfix) with ESMTP id 87FF7180002 for ; Sat, 24 Sep 2022 09:36:36 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 18896B80DA9; Sat, 24 Sep 2022 09:36:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7DCC7C433C1; Sat, 24 Sep 2022 09:36:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1664012193; bh=OUuAuEMHOOoW2UUphtHEcJFzOd5jzmKbeMgag01UxIA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=MaL/dLau/5/WSqjg4siwpxCmhAP5uMcabVH4UVhiJmA5hJU4pY/EMFC2ZLQOUIYt9 nBidURuV0BQ2whZCjUUeB4H3eiNVTkiRLHyBQ66+/vxDre6nLPp3uoDujGWRinvM/t frEK4ii6zJj9MuJEuQkkiW/dKzyIJQIBz/Snf2qSAYQaGq5yAHzEQywEmWBopjDB/+ o1wa0++gwS1fdzdSOsttOI+kmDx4Iu3ZMxipqkhUdMrv4X0/7CGjjCh2dXUwKS+S2r Fx15OFTbQtaYDBm4kTqvTA7XPCeNRfv9UPDVtYO50XtQ2KAQCl1YNKBYybg9px/QkP E6nbHXIsB67rw== Date: Sat, 24 Sep 2022 12:36:12 +0300 From: Mike Rapoport To: "Kirill A. Shutemov" Cc: Tom Lendacky , Dionna Amalie Glaze , Dave Hansen , Mel Gorman , Vlastimil Babka , Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Joerg Roedel , Ard Biesheuvel , Andi Kleen , Kuppuswamy Sathyanarayanan , David Rientjes , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Dario Faggioli , David Hildenbrand , Marcelo Cerri , tim.gardner@canonical.com, Khalid ElMously , philip.cox@canonical.com, the arch/x86 maintainers , Linux Memory Management List , linux-coco@lists.linux.dev, linux-efi , LKML , Mike Rapoport Subject: Re: [PATCHv7 02/14] mm: Add support for unaccepted memory Message-ID: References: <2981e25e-9cda-518a-9750-b8694f2356b5@amd.com> <984e07ed-914f-93ca-a141-3fc8677878e0@intel.com> <20220924010302.bwas4zbro37rrxai@box.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220924010302.bwas4zbro37rrxai@box.shutemov.name> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664012196; a=rsa-sha256; cv=none; b=UcUnws1w1qwtHEvwraj5xhcVfGbfMXMYW1F9QwDfDFmlpxjit4RrC+SggBroa4OEtCGjqF kZUg2Vh7PNprx7CKojLeJnI8YWeVu9bAaRZtIWOVFe14CxXqXJIhONJ3kXdFEioNHfrCoO N9HZS3Dd4NK884Au87Z3k3jkBPApLsM= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="MaL/dLau"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf24.hostedemail.com: domain of rppt@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664012196; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xyTZ5+78MGsw1xEH/HiKnpKSTLPd3LqAFDsXsogVCEE=; b=UPMNP0pq7mxoD7K3ePyOk3eaxKxve47/UJk5UXsEZI08TJHW1MpAeP9Xc3MUwzUtc6cP+z 8xg1TMBlcTbgfiYY9xWoqXVHfrydfTPZB3Whu93BGLywUIHlYICs8UHBcUM9bhQ5qdPe7X 3kKYK3WK5FZubp9ENY3YehthCO3e15Q= Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="MaL/dLau"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf24.hostedemail.com: domain of rppt@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=rppt@kernel.org X-Rspam-User: X-Stat-Signature: 17gw9e41dy77s1t5o5n4k1hs3yxiukzn X-Rspamd-Queue-Id: 87FF7180002 X-Rspamd-Server: rspam09 X-HE-Tag: 1664012196-611753 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Sep 24, 2022 at 04:03:02AM +0300, Kirill A. Shutemov wrote: > On Thu, Sep 22, 2022 at 09:31:12AM -0500, Tom Lendacky wrote: > > On 9/8/22 14:28, Mike Rapoport wrote: > > > On Thu, Sep 08, 2022 at 09:23:07AM -0700, Dionna Amalie Glaze wrote: > > > > > > > > > > Looks like the first access to the memory map fails, although I think > > > > > it's not in INIT_LIST_HEAD() but rather in init_page_count(). > > > > > > > > > > I'd start with making sure that page_alloc::memmap_alloc() actually returns > > > > > accepted memory. If you build kernel with CONFIG_DEBUG_VM=y the memory map > > > > > will poisoned in this function, so my guess is it'd crash there. > > > > > > > > > > > > > That's a wonderful hint, thank you! I did not run this test > > > > CONFIG_DEBUG_VM set, but you think it's possible it could still be > > > > here? > > > > > > It depends on how you configured your kernel. Say, defconfig does not set > > > it. > > > > > > > I also hit the issue at 256GB. My config is using CONFIG_SPARSEMEM_VMEMMAP > > and fails in memmap_init_range() when attempting to add the first PFN. It > > looks like the underlying page that is backing the vmemmap has not been > > accepted (I receive a #VC 0x404 => page not validated). > > > > Kirill, is this a path that you've looked at? It would appear that somewhere > > in the vmemmap_populate_hugepages() path, some memory acceptance needs to be > > done for the pages that are used to back vmemmap. I'm not very familiar with > > this code, so I'm not sure why everything works for a guest with 255GB of > > memory, but then fails for a guest with 256GB of memory. > > Hm. I don't have machine that large at hands at the moment. And I have not > looked at the codepath before. > > I will try to look into the issue. I'd add some printks to verify we actually try to accept the allocated memory. E.g. something like the patch below: diff --git a/arch/x86/mm/unaccepted_memory.c b/arch/x86/mm/unaccepted_memory.c index 9ec2304272dc..8f00639facc4 100644 --- a/arch/x86/mm/unaccepted_memory.c +++ b/arch/x86/mm/unaccepted_memory.c @@ -22,6 +22,9 @@ void accept_memory(phys_addr_t start, phys_addr_t end) if (!boot_params.unaccepted_memory) return; + if (system_state == SYSTEM_BOOTING) + pr_info("%s: start: %pa end: %pa %pS\n", __func__, &start, &end, (void *)_RET_IP_); + bitmap = __va(boot_params.unaccepted_memory); range_start = start / PMD_SIZE; diff --git a/mm/memblock.c b/mm/memblock.c index a1f7f8b304d5..029dd520102d 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1535,7 +1535,7 @@ void * __init memblock_alloc_exact_nid_raw( phys_addr_t min_addr, phys_addr_t max_addr, int nid) { - memblock_dbg("%s: %llu bytes align=0x%llx nid=%d from=%pa max_addr=%pa %pS\n", + pr_info("%s: %llu bytes align=0x%llx nid=%d from=%pa max_addr=%pa %pS\n", __func__, (u64)size, (u64)align, nid, &min_addr, &max_addr, (void *)_RET_IP_); @@ -1567,7 +1567,7 @@ void * __init memblock_alloc_try_nid_raw( phys_addr_t min_addr, phys_addr_t max_addr, int nid) { - memblock_dbg("%s: %llu bytes align=0x%llx nid=%d from=%pa max_addr=%pa %pS\n", + pr_info("%s: %llu bytes align=0x%llx nid=%d from=%pa max_addr=%pa %pS\n", __func__, (u64)size, (u64)align, nid, &min_addr, &max_addr, (void *)_RET_IP_); > -- > Kiryl Shutsemau / Kirill A. Shutemov > -- Sincerely yours, Mike.