From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-f43.google.com (mail-qa0-f43.google.com [209.85.216.43]) by kanga.kvack.org (Postfix) with ESMTP id 5DA2E6B0035 for ; Thu, 27 Mar 2014 16:34:16 -0400 (EDT) Received: by mail-qa0-f43.google.com with SMTP id j15so4337464qaq.2 for ; Thu, 27 Mar 2014 13:34:16 -0700 (PDT) Received: from e8.ny.us.ibm.com (e8.ny.us.ibm.com. [32.97.182.138]) by mx.google.com with ESMTPS id 68si1656019qgk.12.2014.03.27.13.34.15 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 27 Mar 2014 13:34:15 -0700 (PDT) Received: from /spool/local by e8.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 27 Mar 2014 16:34:15 -0400 Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by d01dlp03.pok.ibm.com (Postfix) with ESMTP id DC0C3C90049 for ; Thu, 27 Mar 2014 16:34:08 -0400 (EDT) Received: from d01av04.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by b01cxnp23032.gho.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id s2RKYCeN65863840 for ; Thu, 27 Mar 2014 20:34:12 GMT Received: from d01av04.pok.ibm.com (localhost [127.0.0.1]) by d01av04.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s2RKYCWZ022914 for ; Thu, 27 Mar 2014 16:34:12 -0400 Date: Thu, 27 Mar 2014 13:33:54 -0700 From: Nishanth Aravamudan Subject: Re: Bug in reclaim logic with exhausted nodes? Message-ID: <20140327203354.GA16651@linux.vnet.ibm.com> References: <20140311210614.GB946@linux.vnet.ibm.com> <20140313170127.GE22247@linux.vnet.ibm.com> <20140324230550.GB18778@linux.vnet.ibm.com> <20140325162303.GA29977@linux.vnet.ibm.com> <20140325181010.GB29977@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Christoph Lameter Cc: linux-mm@kvack.org, rientjes@google.com, linuxppc-dev@lists.ozlabs.org, anton@samba.org, mgorman@suse.de Hi Christoph, On 25.03.2014 [13:25:30 -0500], Christoph Lameter wrote: > On Tue, 25 Mar 2014, Nishanth Aravamudan wrote: > > > On power, very early, we find the 16G pages (gpages in the powerpc arch > > code) in the device-tree: > > > > early_setup -> > > early_init_mmu -> > > htab_initialize -> > > htab_init_page_sizes -> > > htab_dt_scan_hugepage_blocks -> > > memblock_reserve > > which marks the memory > > as reserved > > add_gpage > > which saves the address > > off so future calls for > > alloc_bootmem_huge_page() > > > > hugetlb_init -> > > hugetlb_init_hstates -> > > hugetlb_hstate_alloc_pages -> > > alloc_bootmem_huge_page > > > > > Not sure if I understand that correctly. > > > > Basically this is present memory that is "reserved" for the 16GB usage > > per the LPAR configuration. We honor that configuration in Linux based > > upon the contents of the device-tree. It just so happens in the > > configuration from my original e-mail that a consequence of this is that > > a NUMA node has memory (topologically), but none of that memory is free, > > nor will it ever be free. > > Well dont do that > > > Perhaps, in this case, we could just remove that node from the N_MEMORY > > mask? Memory allocations will never succeed from the node, and we can > > never free these 16GB pages. It is really not any different than a > > memoryless node *except* when you are using the 16GB pages. > > That looks to be the correct way to handle things. Maybe mark the node as > offline or somehow not present so that the kernel ignores it. This is a SLUB condition: mm/slub.c::early_kmem_cache_node_alloc(): ... page = new_slab(kmem_cache_node, GFP_NOWAIT, node); ... if (page_to_nid(page) != node) { printk(KERN_ERR "SLUB: Unable to allocate memory from " "node %d\n", node); printk(KERN_ERR "SLUB: Allocating a useless per node structure " "in order to be able to continue\n"); } ... Since this is quite early, and we have not set up the nodemasks yet, does it make sense to perhaps have a temporary init-time nodemask that we set bits in here, and "fix-up" those nodes when we setup the nodemasks? Thanks, Nish -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org