From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christoph Lameter Subject: Re: No, really, stop trying to delete slab until you've finished making slub perform as well Date: Fri, 15 Aug 2008 13:24:24 -0500 Message-ID: <48A5C9D8.2030507@linux-foundation.org> References: <20080805210125.A897.KOSAKI.MOTOHIRO@jp.fujitsu.com> <48986AC6.5030406@linux-foundation.org> <20080813194222.E77F.KOSAKI.MOTOHIRO@jp.fujitsu.com> <48A2DD2C.3090602@linux-foundation.org> <2f11576a0808130714k2cd031c4nd6eea3506831cac9@mail.gmail.com> <48A2F030.9080301@linux-foundation.org> <2f11576a0808130805s6963e2a4mbb82ad7221cf80a2@mail.gmail.com> <48A48B00.1090703@linux-foundation.org> <2f11576a0808150944g678f1b8di1c4d50de7358b306@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Matthew Wilcox , Pekka Enberg , akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Mel Gorman , andi@firstfloor.org, Rik van Riel To: KOSAKI Motohiro Return-path: Received: from smtp1.linux-foundation.org ([140.211.169.13]:42836 "EHLO smtp1.linux-foundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757244AbYHOSZo (ORCPT ); Fri, 15 Aug 2008 14:25:44 -0400 In-Reply-To: <2f11576a0808150944g678f1b8di1c4d50de7358b306@mail.gmail.com> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: KOSAKI Motohiro wrote: >> Then we have another mechanism that will avoid >> off node memory references by allocating new slabs. Can you set the >> node_defrag parameter to 0? (Noted by Adrian). > > Please let me know that operations ? The control over the preferences of node local vs. remote defrag is occurring via /sys/kernel/slab//remote_node_defrag ratio. Default is 10%. Comments in get_any_partial explain the operations. The default setting means that in 9 out of 10 cases slub will prefer creating a new slab over taking one from the remote node (meaning the memory is node local, probably not important in your 2 node case). It will therefore waste memory because local memory may be more efficient to use. Setting remote_node_defrag_ratio to 100 will make slub always take the remote slab instead of allocating a new one. /* * The defrag ratio allows a configuration of the tradeoffs between * inter node defragmentation and node local allocations. A lower * defrag_ratio increases the tendency to do local allocations * instead of attempting to obtain partial slabs from other nodes. * * If the defrag_ratio is set to 0 then kmalloc() always * returns node local objects. If the ratio is higher then kmalloc() * may return off node objects because partial slabs are obtained * from other nodes and filled up. * * If /sys/kernel/slab/xx/defrag_ratio is set to 100 (which makes * defrag_ratio = 1000) then every (well almost) allocation will * first attempt to defrag slab caches on other nodes. This means * scanning over all nodes to look for partial slabs which may be * expensive if we do it every time we are trying to find a slab * with available objects. */ if (!s->remote_node_defrag_ratio || get_cycles() % 1024 > s->remote_node_defrag_ratio) return NULL;