From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754230Ab0I1MpA (ORCPT ); Tue, 28 Sep 2010 08:45:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41819 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754054Ab0I1Mo6 (ORCPT ); Tue, 28 Sep 2010 08:44:58 -0400 Message-ID: <4CA1E338.6070201@redhat.com> Date: Tue, 28 Sep 2010 14:44:40 +0200 From: Avi Kivity User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc13 Lightning/1.0b3pre Thunderbird/3.1.4 MIME-Version: 1.0 To: Nathan Fontenot CC: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@ozlabs.org, Greg KH , Dave Hansen , KAMEZAWA Hiroyuki Subject: Re: [PATCH 0/8] v2 De-Couple sysfs memory directories from memory sections References: <4CA0EBEB.1030204@austin.ibm.com> In-Reply-To: <4CA0EBEB.1030204@austin.ibm.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/27/2010 09:09 PM, Nathan Fontenot wrote: > This set of patches decouples the concept that a single memory > section corresponds to a single directory in > /sys/devices/system/memory/. On systems > with large amounts of memory (1+ TB) there are perfomance issues > related to creating the large number of sysfs directories. For > a powerpc machine with 1 TB of memory we are creating 63,000+ > directories. This is resulting in boot times of around 45-50 > minutes for systems with 1 TB of memory and 8 hours for systems > with 2 TB of memory. With this patch set applied I am now seeing > boot times of 5 minutes or less. > > The root of this issue is in sysfs directory creation. Every time > a directory is created a string compare is done against all sibling > directories to ensure we do not create duplicates. The list of > directory nodes in sysfs is kept as an unsorted list which results > in this being an exponentially longer operation as the number of > directories are created. > > The solution solved by this patch set is to allow a single > directory in sysfs to span multiple memory sections. This is > controlled by an optional architecturally defined function > memory_block_size_bytes(). The default definition of this > routine returns a memory block size equal to the memory section > size. This maintains the current layout of sysfs memory > directories as it appears to userspace to remain the same as it > is today. > Why not update sysfs directory creation to be fast, for example by using an rbtree instead of a linked list. This fixes an implementation problem in the kernel instead of working around it and creating a new ABI. New ABIs mean old tools won't work, and new tools need to understand both ABIs. -- error compiling committee.c: too many arguments to function