All of lore.kernel.org
 help / color / mirror / Atom feed
From: Colin Walters <walters@verbum.org>
To: Patches and discussions about the oe-core layer
	<openembedded-core@lists.openembedded.org>
Cc: bitbake-devel <bitbake-devel@lists.openembedded.org>
Subject: Re: [OE-core] Bitbake Memory Usage
Date: Mon, 20 Feb 2012 09:05:00 -0500	[thread overview]
Message-ID: <1329746700.22425.2.camel@lenny> (raw)
In-Reply-To: <1329608182.4436.48.camel@ted>

On Sat, 2012-02-18 at 23:36 +0000, Richard Purdie wrote:
> As soon as the child starts trying to remove
> things from memory, we lose the benefits of CoW and USS and PSS rise.

Note that even leaving out the garbage collector, the cPython VM
incrementing/decrementing the refcount of objects will force the copy.

The python multiprocessing module has some classes which allow
explicitly sharing memory.  Certainly for various caches e.g. if
they're just read-only after a certain point it might make sense
to wrap them in e.g. a multiprocessing.Value.





WARNING: multiple messages have this Message-ID (diff)
From: Colin Walters <walters@verbum.org>
To: Patches and discussions about the oe-core layer
	<openembedded-core@lists.openembedded.org>
Cc: bitbake-devel <bitbake-devel@lists.openembedded.org>
Subject: Re: Bitbake Memory Usage
Date: Mon, 20 Feb 2012 09:05:00 -0500	[thread overview]
Message-ID: <1329746700.22425.2.camel@lenny> (raw)
In-Reply-To: <1329608182.4436.48.camel@ted>

On Sat, 2012-02-18 at 23:36 +0000, Richard Purdie wrote:
> As soon as the child starts trying to remove
> things from memory, we lose the benefits of CoW and USS and PSS rise.

Note that even leaving out the garbage collector, the cPython VM
incrementing/decrementing the refcount of objects will force the copy.

The python multiprocessing module has some classes which allow
explicitly sharing memory.  Certainly for various caches e.g. if
they're just read-only after a certain point it might make sense
to wrap them in e.g. a multiprocessing.Value.





  parent reply	other threads:[~2012-02-20 19:31 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-02-18 23:36 Bitbake Memory Usage Richard Purdie
2012-02-19  2:55 ` Richard Purdie
2012-02-19  2:55   ` [bitbake-devel] " Richard Purdie
2012-02-19  8:34   ` [OE-core] " Phil Blundell
2012-02-19  8:34     ` [bitbake-devel] " Phil Blundell
2012-02-19 15:41     ` [OE-core] " Richard Purdie
2012-02-19 15:41       ` [bitbake-devel] " Richard Purdie
2012-02-19 16:04       ` [OE-core] " Phil Blundell
2012-02-19 16:04         ` [bitbake-devel] " Phil Blundell
2012-02-20 14:05 ` Colin Walters [this message]
2012-02-20 14:05   ` Colin Walters
2012-02-20 19:57   ` [OE-core] " Chris Larson
2012-02-20 19:57     ` Chris Larson
2012-02-21 22:15     ` [OE-core] " Richard Purdie
2012-02-21 22:15       ` [bitbake-devel] " Richard Purdie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1329746700.22425.2.camel@lenny \
    --to=walters@verbum.org \
    --cc=bitbake-devel@lists.openembedded.org \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.