All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Gmane Admin" <gley-yocto@m.gmane-mx.org>
To: yocto@lists.yoctoproject.org
Subject: bitbake controlling memory use
Date: Sun, 11 Apr 2021 17:23:17 +0200	[thread overview]
Message-ID: <s4v495$bak$1@ciao.gmane.io> (raw)

My build machine has 8 cores + HT so bitbake enthusiastically assumes 
16. However I have (only?) 16GB of RAM (+24GB swap space).

16GB of RAM has always been more then enough with 4 core + HT, but now 
building certain recipes (nodejs, but rust will probably do it as well) 
eats up more memory then there actually is RAM causing excessive swapping.

In fact the swapping is so excessive that just this single recipe 
determines largely the total image build time. However restricting 
bitbake (or actually parallel make) to use only 4 cores + HT sounds also 
like a waste.

I know this has been looked at in the past, but I think it needs 
fundamental resolving (other then, hé, why not add just another stick of 
memory).

What I do manually when I run into swapping so bad my desktop becomes 
unresponsive is ssh from another machine and then stop (not kill) the 
processes that use the most memory.

These then get swapped out, but not back in, allowing the remaining 
tasks to complete without swapping. Then when sufficient memory becomes 
free again I continue the stopped processes.

Isn't this something that could be added to bitbake to automate using 
memory efficiently?


             reply	other threads:[~2021-04-11 15:23 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <1b153bce-a66a-45ee-a5c6-963ea6fb1c82.949ef384-8293-46b8-903f-40a477c056ae.12d167de-8268-4441-bd03-b78a3e04713e@emailsignatures365.codetwo.com>
     [not found] ` <1b153bce-a66a-45ee-a5c6-963ea6fb1c82.0d2bd5fa-15cc-4b27-b94e-83614f9e5b38.a8aa18ca-4a64-4525-851c-c6d34f355f1f@emailsignatures365.codetwo.com>
2021-04-11 15:23   ` Gmane Admin [this message]
2021-04-11 15:27     ` [yocto] bitbake controlling memory use Alexander Kanavin
2021-04-11 15:49       ` Gmane Admin
2021-04-11 15:55         ` [yocto] " Alexander Kanavin
2021-04-11 16:08           ` Gmane Admin
2021-04-11 16:19             ` [yocto] " Alexander Kanavin
2021-04-14  1:14               ` Randy MacLeod
2021-04-14  3:14                 ` Khem Raj
2021-04-14  4:59                 ` Richard Purdie
2021-04-17 22:17                   ` Gmane Admin
2021-04-18  9:59                     ` [yocto] " Richard Purdie
2021-04-18 19:34                       ` Gmane Admin
     [not found]                       ` <16770AD5E94DEAE5.1089@lists.yoctoproject.org>
2022-12-24 18:58                         ` Ferry Toth
2022-12-26  2:11                           ` [yocto] " Randy MacLeod
2022-12-27 16:56                             ` Ferry Toth
2021-06-05 13:35                   ` Gmane Admin
2021-06-08 19:08                     ` [yocto] " Trevor Gamblin
2021-06-10  7:09                       ` Gmane Admin
     [not found]                   ` <1685B30F5819A655.13459@lists.yoctoproject.org>
2021-06-07 19:27                     ` Gmane Admin
2021-06-08 19:12                       ` [yocto] " Trevor Gamblin
2021-06-10  9:22                       ` Ferry Toth
2021-06-10 19:06                         ` [yocto] " Trevor Gamblin
2021-06-10 20:35                           ` Ferry Toth
2021-06-12 16:31                             ` Ferry Toth
2021-06-13  0:38                               ` Randy MacLeod
2021-06-13 10:58                                 ` Ferry Toth
2023-01-03 14:15                                 ` Ferry Toth
2023-01-03 14:18                                   ` [yocto] " Alexander Kanavin
2023-01-03 14:29                                     ` Ferry Toth
2023-01-03 14:36                                       ` Quentin Schulz
2023-01-03 14:41                                         ` Ferry Toth
2023-01-03 15:03                                           ` Janne Kiiskila
2023-01-03 15:12                                             ` Ferry Toth
2023-01-03 15:24                                               ` Alexander Kanavin
     [not found]                                               ` <1736D5DCC66BC3DE.4716@lists.yoctoproject.org>
2023-01-03 15:37                                                 ` Alexander Kanavin
2023-01-03 15:58                                                   ` Ferry Toth
2023-01-03 17:33                                                     ` Alexander Kanavin
2023-01-08 22:13                                                       ` Ferry Toth
2023-01-09 10:39                                                         ` Alexander Kanavin
2023-01-09 10:43                                                           ` Richard Purdie
2023-01-09 10:54                                                             ` Ferry Toth
2023-01-19 19:59                                                               ` Ferry Toth
2023-01-09 10:49                                                           ` Ferry Toth
2023-01-10  1:57                                                             ` Randy MacLeod
2023-01-10  9:24                                                               ` Ferry Toth
2023-01-14 22:33                                                                 ` Ferry Toth
2023-08-28 12:56                                                                   ` Martin Hundebøll
2023-01-03 15:44                                       ` Martin Jansa
2021-04-12  2:25     ` Chen Qi
2021-04-12 18:45       ` Gmane Admin
2021-04-12  8:13     ` [yocto] " Robert Berger
2021-04-12 18:47       ` Gmane Admin
2021-04-12  8:28     ` [yocto] " Mike Looijmans

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='s4v495$bak$1@ciao.gmane.io' \
    --to=gley-yocto@m.gmane-mx.org \
    --cc=yocto@lists.yoctoproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.