All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dave Hansen <dave@linux.vnet.ibm.com>
To: Greg KH <greg@kroah.com>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Robin Holt <holt@sgi.com>,
	linuxppc-dev@lists.ozlabs.org,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Subject: Re: [PATCH 0/4] De-couple sysfs memory directories from memory sections
Date: Thu, 20 Jan 2011 09:09:01 -0800	[thread overview]
Message-ID: <1295543341.9039.588.camel@nimitz> (raw)
In-Reply-To: <20110120164555.GA30922@kroah.com>

On Thu, 2011-01-20 at 08:45 -0800, Greg KH wrote:
> On Thu, Jan 20, 2011 at 10:36:40AM -0600, Nathan Fontenot wrote:
> > The root of this issue is in sysfs directory creation. Every time
> > a directory is created a string compare is done against sibling
> > directories ( see sysfs_find_dirent() ) to ensure we do not create 
> > duplicates.  The list of directory nodes in sysfs is kept as an
> > unsorted list which results in this being an exponentially longer
> > operation as the number of directories are created.
> 
> Again, are you sure about this?  I thought we resolved this issue in the
> past, but you were going to check it.  Did you?

Just to be clear, simply reducing the number of kobjects can make these
patches worthwhile on their own.  I originally figured that the
SECTION_SIZE would go up over time as systems got larger, and _that_
would keep the number of sections and number of sysfs objects down.
Well, that turned out to be wrong, and we're eating up a ton of memory
now.  We can't fix the SECTION_SIZE easily, but we can reduce the number
of kobjects that we need to track the sections.  *That* is the main
benefit I see from these patches.

I think there's a problem worth fixing, even ignoring the directory
creation issue (if it still exists).

-- Dave

WARNING: multiple messages have this Message-ID (diff)
From: Dave Hansen <dave@linux.vnet.ibm.com>
To: Greg KH <greg@kroah.com>
Cc: Nathan Fontenot <nfont@austin.ibm.com>,
	linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org,
	linux-kernel@vger.kernel.org,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Robin Holt <holt@sgi.com>
Subject: Re: [PATCH 0/4] De-couple sysfs memory directories from memory sections
Date: Thu, 20 Jan 2011 09:09:01 -0800	[thread overview]
Message-ID: <1295543341.9039.588.camel@nimitz> (raw)
In-Reply-To: <20110120164555.GA30922@kroah.com>

On Thu, 2011-01-20 at 08:45 -0800, Greg KH wrote:
> On Thu, Jan 20, 2011 at 10:36:40AM -0600, Nathan Fontenot wrote:
> > The root of this issue is in sysfs directory creation. Every time
> > a directory is created a string compare is done against sibling
> > directories ( see sysfs_find_dirent() ) to ensure we do not create 
> > duplicates.  The list of directory nodes in sysfs is kept as an
> > unsorted list which results in this being an exponentially longer
> > operation as the number of directories are created.
> 
> Again, are you sure about this?  I thought we resolved this issue in the
> past, but you were going to check it.  Did you?

Just to be clear, simply reducing the number of kobjects can make these
patches worthwhile on their own.  I originally figured that the
SECTION_SIZE would go up over time as systems got larger, and _that_
would keep the number of sections and number of sysfs objects down.
Well, that turned out to be wrong, and we're eating up a ton of memory
now.  We can't fix the SECTION_SIZE easily, but we can reduce the number
of kobjects that we need to track the sections.  *That* is the main
benefit I see from these patches.

I think there's a problem worth fixing, even ignoring the directory
creation issue (if it still exists).

-- Dave


WARNING: multiple messages have this Message-ID (diff)
From: Dave Hansen <dave@linux.vnet.ibm.com>
To: Greg KH <greg@kroah.com>
Cc: Nathan Fontenot <nfont@austin.ibm.com>,
	linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org,
	linux-kernel@vger.kernel.org,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Robin Holt <holt@sgi.com>
Subject: Re: [PATCH 0/4] De-couple sysfs memory directories from memory sections
Date: Thu, 20 Jan 2011 09:09:01 -0800	[thread overview]
Message-ID: <1295543341.9039.588.camel@nimitz> (raw)
In-Reply-To: <20110120164555.GA30922@kroah.com>

On Thu, 2011-01-20 at 08:45 -0800, Greg KH wrote:
> On Thu, Jan 20, 2011 at 10:36:40AM -0600, Nathan Fontenot wrote:
> > The root of this issue is in sysfs directory creation. Every time
> > a directory is created a string compare is done against sibling
> > directories ( see sysfs_find_dirent() ) to ensure we do not create 
> > duplicates.  The list of directory nodes in sysfs is kept as an
> > unsorted list which results in this being an exponentially longer
> > operation as the number of directories are created.
> 
> Again, are you sure about this?  I thought we resolved this issue in the
> past, but you were going to check it.  Did you?

Just to be clear, simply reducing the number of kobjects can make these
patches worthwhile on their own.  I originally figured that the
SECTION_SIZE would go up over time as systems got larger, and _that_
would keep the number of sections and number of sysfs objects down.
Well, that turned out to be wrong, and we're eating up a ton of memory
now.  We can't fix the SECTION_SIZE easily, but we can reduce the number
of kobjects that we need to track the sections.  *That* is the main
benefit I see from these patches.

I think there's a problem worth fixing, even ignoring the directory
creation issue (if it still exists).

-- Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2011-01-20 17:09 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-20 16:36 [PATCH 0/4] De-couple sysfs memory directories from memory sections Nathan Fontenot
2011-01-20 16:36 ` Nathan Fontenot
2011-01-20 16:43 ` [PATCH 1/4] Allow memory blocks to span multiple " Nathan Fontenot
2011-01-20 16:43   ` Nathan Fontenot
2011-01-20 16:44 ` [PATCH 2/4] Update phys_index to [start|end]_section_nr Nathan Fontenot
2011-01-20 16:44   ` Nathan Fontenot
2011-01-20 16:45 ` [PATCH 3/4]Define memory_block_size_bytes for powerpc/pseries Nathan Fontenot
2011-01-20 16:45   ` Nathan Fontenot
2011-02-06 23:39   ` Benjamin Herrenschmidt
2011-02-06 23:39     ` Benjamin Herrenschmidt
2011-02-06 23:39     ` Benjamin Herrenschmidt
2011-02-07  1:42     ` Greg KH
2011-02-07  1:42       ` Greg KH
2011-02-07  1:42       ` Greg KH
2011-01-20 16:45 ` [PATCH 0/4] De-couple sysfs memory directories from memory sections Greg KH
2011-01-20 16:45   ` Greg KH
2011-01-20 16:51   ` Nathan Fontenot
2011-01-20 16:51     ` Nathan Fontenot
2011-01-20 17:25     ` Greg KH
2011-01-20 17:25       ` Greg KH
2011-01-20 17:09   ` Dave Hansen [this message]
2011-01-20 17:09     ` Dave Hansen
2011-01-20 17:09     ` Dave Hansen
2011-01-20 16:46 ` [PATCH 4/4] Define memory_block_size_bytes for x86_64 with CONFIG_X86_UV Nathan Fontenot
2011-01-20 16:46   ` Nathan Fontenot
  -- strict thread matches above, loose matches on Subject: below --
2011-01-10 18:08 [PATCH 0/4] De-couple sysfs memory directories from memory sections Nathan Fontenot
2011-01-10 18:08 ` Nathan Fontenot
2011-01-10 18:08 ` Nathan Fontenot
2011-01-10 18:44 ` Greg KH
2011-01-10 18:44   ` Greg KH
2011-01-10 18:44   ` Greg KH
2011-01-10 18:47   ` Nathan Fontenot
2011-01-10 18:47     ` Nathan Fontenot
2011-01-10 18:47     ` Nathan Fontenot
2011-01-10 19:11     ` Robin Holt
2011-01-10 19:11       ` Robin Holt
2011-01-10 19:11       ` Robin Holt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1295543341.9039.588.camel@nimitz \
    --to=dave@linux.vnet.ibm.com \
    --cc=greg@kroah.com \
    --cc=holt@sgi.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.