From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C6FEC433FE for ; Thu, 3 Dec 2020 18:35:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E3CA207A9 for ; Thu, 3 Dec 2020 18:35:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E3CA207A9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E1BDD6B006C; Thu, 3 Dec 2020 13:35:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DCAE76B006E; Thu, 3 Dec 2020 13:35:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CBA466B0072; Thu, 3 Dec 2020 13:35:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0058.hostedemail.com [216.40.44.58]) by kanga.kvack.org (Postfix) with ESMTP id B288E6B006C for ; Thu, 3 Dec 2020 13:35:03 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7D3ED21E4 for ; Thu, 3 Dec 2020 18:35:03 +0000 (UTC) X-FDA: 77552822886.21.girls07_370b1b2273be Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 62E2D180442C0 for ; Thu, 3 Dec 2020 18:35:03 +0000 (UTC) X-HE-Tag: girls07_370b1b2273be X-Filterd-Recvd-Size: 5335 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Thu, 3 Dec 2020 18:35:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607020502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJUw/pr4nf2NM2FpsqPAixF+QB/rQYSePgBJ4oHqJXc=; b=S0ZeWodFgwP5KKM7XcWYRUySeXJtERAQrbG0FSLeO/aTtnWbPGJWI0Zs6HT2qgC5Q/mZA9 e32j0ce0J5pq2Fd9CXhCkd0JZBnsiRuSMJtFM4DVCgmd70Mr9DYw3Cy6Wz/ZPpOz4TQ+VT P3QMeL7zqIEWOzFe/rMDHxRXGoMclf0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-275-JJKstRQZO3ivnDBBixqMBA-1; Thu, 03 Dec 2020 13:34:58 -0500 X-MC-Unique: JJKstRQZO3ivnDBBixqMBA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5B874AFA81; Thu, 3 Dec 2020 18:34:56 +0000 (UTC) Received: from mail (ovpn-112-118.rdu2.redhat.com [10.10.112.118]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A33395D6BA; Thu, 3 Dec 2020 18:34:52 +0000 (UTC) Date: Thu, 3 Dec 2020 13:34:52 -0500 From: Andrea Arcangeli To: Mike Rapoport Cc: Andrew Morton , linux-mm@kvack.org, Baoquan He , David Hildenbrand , Mel Gorman , Michal Hocko , Mike Rapoport , Qian Cai , Vlastimil Babka , linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm: refactor initialization of stuct page for holes in memory layout Message-ID: References: <20201201181502.2340-1-rppt@kernel.org> <20201202154736.5799e01b4c27a75b98e863fc@linux-foundation.org> <20201203062549.GG751215@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201203062549.GG751215@kernel.org> User-Agent: Mutt/2.0.2 (2020-11-20) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello, On Thu, Dec 03, 2020 at 08:25:49AM +0200, Mike Rapoport wrote: > On Wed, Dec 02, 2020 at 03:47:36PM -0800, Andrew Morton wrote: > > On Tue, 1 Dec 2020 20:15:02 +0200 Mike Rapoport wrote: > > > > > From: Mike Rapoport > > > > > > There could be struct pages that are not backed by actual physical memory. > > > This can happen when the actual memory bank is not a multiple of > > > SECTION_SIZE or when an architecture does not register memory holes > > > reserved by the firmware as memblock.memory. > > > > > > Such pages are currently initialized using init_unavailable_mem() function > > > that iterated through PFNs in holes in memblock.memory and if there is a > > > struct page corresponding to a PFN, the fields if this page are set to > > > default values and it is marked as Reserved. > > > > > > init_unavailable_mem() does not take into account zone and node the page > > > belongs to and sets both zone and node links in struct page to zero. > > > > > > On a system that has firmware reserved holes in a zone above ZONE_DMA, for > > > instance in a configuration below: > > > > > > # grep -A1 E820 /proc/iomem > > > 7a17b000-7a216fff : Unknown E820 type > > > 7a217000-7bffffff : System RAM > > > > > > unset zone link in struct page will trigger > > > > > > VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); > > > > That sounds pretty serious. My understanding is that with DEBUG_VM=n the invariant that broke won't cause trouble, but Fedora is helping the upstream testing by keeping DEBUG_VM=y and it's shipping with v5.8 and v5.9 for a while, so it could very well crash those kernels if they've that type 20 thing in the e820 map. > > > > > because there are pages in both ZONE_DMA32 and ZONE_DMA (unset zone link in > > > struct page) in the same pageblock. > > > > > > Interleave initialization of pages that correspond to holes with the > > > initialization of memory map, so that zone and node information will be > > > properly set on such pages. > > > > > > > Should this be backported to -stable? If so, do we have a suitable Fixes:? > > Sorry, I forgot to add > > Fixes: 73a6e474cb37 ("mm: memmap_init: iterate over memblock regions rather that check each PFN") I've been wondering already why I'm the only one getting a crash every two weeks. Ince it crashed in MADV_HUGEPAGE of qemu that would definitely happened even with Fedora despite the THP enabled = madvise, and it hung qemu for good so it was noticeable since it was in direction compaction. Other times it was in kcompactd so it just killed the kernel thread and it was only noticeable in the kernel logs and probably it doesn't happen that frequently unless THP enabled = always, although it could still happen, compaction isn't used just for THP. Thanks, Andrea