Limiting Yocto Memory Usage

Limiting Yocto Memory Usage

[Luis Merayo]

[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]

Hi,

I would like to know if there is a way to limit Bitbake's Memory usage to a
specific percentage of the available RAM. For example, can the Yocto build
be limited to use a maximum of 70% of RAM?

Could these three variables achieve this?

 BB_PRESSURE_MAX_CPU
<https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_CPU>
, BB_PRESSURE_MAX_IO
<https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_IO>
 , BB_PRESSURE_MAX_MEMORY
<https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_MEMORY>

I see the range for these variables is from 1 to 1000000, so I would like
to understand if there is a way to specify their values as a function of
the available RAM.

Regards,

Luis

-- 
 <https://ridgerun.ai/>
This email and any attachments are intended for the 
sole use of the named recipient(s) and contain(s) confidential information 
that may be proprietary, privileged, or copyrighted under applicable law. 
If you are not the intended recipient, do not read, copy, or forward this 
email message or any attachments, delete this email message and any 
attachments immediately.

[-- Attachment #2: Type: text/html, Size: 4415 bytes --]

Re: [yocto] Limiting Yocto Memory Usage

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 2847 bytes --]

Hi Luis,

On 2026-07-10 17:26, Luis Merayo via lists.yoctoproject.org wrote:
> Hi,
>
> I would like to know if there is a way to limit Bitbake's Memory usage 
> to a specific percentage of the available RAM. For example, can the 
> Yocto build be limited to use a maximum of 70% of RAM?
Not without using a container to do the build.
>
> Could these three variables achieve this?
>
> BB_PRESSURE_MAX_CPU 
> <https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_CPU>, 
> BB_PRESSURE_MAX_IO 
> <https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_IO> , 
> BB_PRESSURE_MAX_MEMORY 
> <https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_MEMORY>
>
> I see the range for these variables is from 1 to 1000000, so I would 
> like to understand if there is a way to specify their values as a 
> function of the available RAM.
No, there is only pressure when resources are over-commited so if you 
have free memory, there's no over-commit.
https://docs.kernel.org/accounting/psi.html

That's the design intent of the bitbake PRESSURE regulation system.

There are plans to work on additional build system regulation.

The first idea is to use the jobserver design:
https://lore.kernel.org/openembedded-core/?q=GNU+AND+jobserver++AND+f%3Amacleod
and that would indirectly limit memory consumption since there would be 
fewer instances of gcc / rustc / ... running
at once.

There's also an even less clear idea about being able to restrict the 
build by specifying memory usage.
That shouldn't be too hard to implement in bitbake in the same point 
where the pressure checks happen.
Are you interested in working on that ?

There are of course other levers to pull to reduce CPU/memory consumption:
https://docs.yoctoproject.org/dev-manual/limiting-resources.html

Have you tried them ?


>
> Regards,
>
> Luis
>
>
>
> File:RidgeRun.ai banner.png <https://ridgerun.ai/>
> This email and any attachments are intended for the sole use of the 
> named recipient(s) and contain(s) confidential information that may be 
> proprietary, privileged, or copyrighted under applicable law. If you 
> are not the intended recipient, do not read, copy, or forward this 
> email message or any attachments, delete this email message and any 
> attachments immediately.
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#66637):https://lists.yoctoproject.org/g/yocto/message/66637
> Mute This Topic:https://lists.yoctoproject.org/mt/120213825/3616765
> Group Owner:yocto+owner@lists.yoctoproject.org
> Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 8837 bytes --]

Re: [yocto] Limiting Yocto Memory Usage

[Ulrich Ölmann]

Hey Luis,

On Fri, Jul 10 2026 at 15:26 -0600, "Luis Merayo via lists.yoctoproject.org" <luis.merayo=ridgerun.com@lists.yoctoproject.org> wrote:
> Hi,
>
> I would like to know if there is a way to limit Bitbake's Memory usage
> to a specific percentage of the available RAM. For example, can the
> Yocto build be limited to use a maximum of 70% of RAM?

if you are doing your Bitbake work on a machine running a systemd based
Linux distribution you could simply do something like

  $ systemd-run \
       --user \
       --scope \
       --property=CPUWeight=<YOUR_CPUWEIGHT> \
       --property=IOWeight=<YOUR_IOWEIGHT> \
       --property=MemoryHigh=<YOUR_MEMORYHIGH> \
       --property=OOMPolicy=kill \
       --unit="bitbake-run" \
       bitbake [...]

to somewhat limit the resources, see systemd.resource-control(5) and
systemd.scope(5) for details regarding the used parameters.

Best regards,
Ulrich


> Could these three variables achieve this?
>
>  BB_PRESSURE_MAX_CPU, BB_PRESSURE_MAX_IO , BB_PRESSURE_MAX_MEMORY
>
> I see the range for these variables is from 1 to 1000000, so I would
> like to understand if there is a way to specify their values as a
> function of the available RAM.
>
> Regards,
>
> Luis
>
>
> This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential
> information that may be proprietary, privileged, or copyrighted under applicable law. If you are not the intended
> recipient, do not read, copy, or forward this email message or any attachments, delete this email message and any
> attachments immediately.
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#66637): https://lists.yoctoproject.org/g/yocto/message/66637
> Mute This Topic: https://lists.yoctoproject.org/mt/120213825/3618626
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [u.oelmann@pengutronix.de]
> -=-=-=-=-=-=-=-=-=-=-=-
-- 
Pengutronix e.K.                           | Ulrich Ölmann               |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [yocto] Limiting Yocto Memory Usage

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 2799 bytes --]

On 2026-07-16 04:22, Ulrich Ölmann via lists.yoctoproject.org wrote:
> Hey Luis,
>
> On Fri, Jul 10 2026 at 15:26 -0600, "Luis Merayo via lists.yoctoproject.org"<luis.merayo=ridgerun.com@lists.yoctoproject.org> wrote:
>> Hi,
>>
>> I would like to know if there is a way to limit Bitbake's Memory usage
>> to a specific percentage of the available RAM. For example, can the
>> Yocto build be limited to use a maximum of 70% of RAM?
> if you are doing your Bitbake work on a machine running a systemd based
> Linux distribution you could simply do something like
>
>    $ systemd-run \
>         --user \
>         --scope \
>         --property=CPUWeight=<YOUR_CPUWEIGHT> \
>         --property=IOWeight=<YOUR_IOWEIGHT> \
>         --property=MemoryHigh=<YOUR_MEMORYHIGH> \
>         --property=OOMPolicy=kill \
>         --unit="bitbake-run" \
>         bitbake [...]
>
> to somewhat limit the resources, see systemd.resource-control(5) and
> systemd.scope(5) for details regarding the used parameters.
>
> Best regards,
> Ulrich


Hi Ulrich,

Using systemd-run is something that I've been meaning to try so it's 
good to hear that you're doing it.

Do you have any experiences to share about this approach ?
Do you do world builds?
Do you run more than one systemd-run build on a given machine?


Of course it doesn't prevent bitbake from pushing the cgroup contained 
system
harder that it can handle and bitbake doesn't have any feedback so 
that's something
to consider when doing the jobserver or free memory build regulation work.

../Randy


>
>
>> Could these three variables achieve this?
>>
>>   BB_PRESSURE_MAX_CPU, BB_PRESSURE_MAX_IO , BB_PRESSURE_MAX_MEMORY
>>
>> I see the range for these variables is from 1 to 1000000, so I would
>> like to understand if there is a way to specify their values as a
>> function of the available RAM.
>>
>> Regards,
>>
>> Luis
>>
>>
>> This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential
>> information that may be proprietary, privileged, or copyrighted under applicable law. If you are not the intended
>> recipient, do not read, copy, or forward this email message or any attachments, delete this email message and any
>> attachments immediately.
>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#66645):https://lists.yoctoproject.org/g/yocto/message/66645
>> Mute This Topic:https://lists.yoctoproject.org/mt/120272854/3616765
>> Group Owner:yocto+owner@lists.yoctoproject.org
>> Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub [randy.macleod@windriver.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 4403 bytes --]

Re: [yocto] Limiting Yocto Memory Usage

[Nate Kent]

[-- Attachment #1: Type: text/plain, Size: 2674 bytes --]

On Tue, Jul 21, 2026 at 05:22 PM, Randy MacLeod wrote:

>
> On 2026-07-16 04:22, Ulrich Ölmann via lists.yoctoproject.org wrote:
> > Hey Luis,
> >
> > On Fri, Jul 10 2026 at 15:26 -0600, "Luis Merayo via
> lists.yoctoproject.org"<luis.merayo=ridgerun.com@lists.yoctoproject.org>
> wrote:
> >> Hi,
> >>
> >> I would like to know if there is a way to limit Bitbake's Memory usage
> >> to a specific percentage of the available RAM. For example, can the
> >> Yocto build be limited to use a maximum of 70% of RAM?
> > if you are doing your Bitbake work on a machine running a systemd based
> > Linux distribution you could simply do something like
> >
> >    $ systemd-run \
> >         --user \
> >         --scope \
> >         --property=CPUWeight=<YOUR_CPUWEIGHT> \
> >         --property=IOWeight=<YOUR_IOWEIGHT> \
> >         --property=MemoryHigh=<YOUR_MEMORYHIGH> \
> >         --property=OOMPolicy=kill \
> >         --unit="bitbake-run" \
> >         bitbake [...]
> >
> > to somewhat limit the resources, see systemd.resource-control(5) and
> > systemd.scope(5) for details regarding the used parameters.
> >
> > Best regards,
> > Ulrich
> 
> 
> Hi Ulrich,
> 
> Using systemd-run is something that I've been meaning to try so it's 
> good to hear that you're doing it.
> 
> Do you have any experiences to share about this approach ?
> Do you do world builds?
> Do you run more than one systemd-run build on a given machine?

I do this as a regular part of my development, though I tend to use '-S' ('-shell', essentially interactive mode) rather than spawning a unit just for a single build. I'll also often modify the resource constraints while BitBake is running if I need to reserve additional resources for non-BitBake tasks while a build is running. That being said, I generally do not do more than a single build at a time simply because reasoning about how to get two competing processes to maximize the usage of the system is a much more difficult challenge.

For theor the most my experience shows that it will work as you expect. If you give the slice three cores and it will act like building on a machine with just three cores, give it a low CPU weight and you'll get a build that's low priority to the scheduler, etc.

If you have other questions I can try to answer them but, for the most part, it's so painless and unsurprising that I don't really have much in the way of advice.

- Nate

PS: I'm sending this from the Yocto Mailing list website and I can't tell if it's going to send as "Github Flavored Markdown" + HTML or the plaintext that I'm typing. Apologies if this comes through mangled. 

[-- Attachment #2: Type: text/html, Size: 2824 bytes --]

Re: [yocto] Limiting Yocto Memory Usage

[Ulrich Ölmann]

Hi Randy,

On Tue, Jul 21 2026 at 11:22 -0400, Randy MacLeod <randy.macleod@windriver.com> wrote:
> On 2026-07-16 04:22, Ulrich Ölmann via lists.yoctoproject.org wrote:
> >
> > Hey Luis,
> >
> > On Fri, Jul 10 2026 at 15:26 -0600, "Luis Merayo via lists.yoctoproject.org"<luis.merayo=ridgerun.com@lists.yoctoproject.org> wrote:
> >>
> >> Hi,
> >>
> >> I would like to know if there is a way to limit Bitbake's Memory usage
> >> to a specific percentage of the available RAM. For example, can the
> >> Yocto build be limited to use a maximum of 70% of RAM?
> > if you are doing your Bitbake work on a machine running a systemd based
> > Linux distribution you could simply do something like
> >
> >    $ systemd-run \
> >         --user \
> >         --scope \
> >         --property=CPUWeight=<YOUR_CPUWEIGHT> \
> >         --property=IOWeight=<YOUR_IOWEIGHT> \
> >         --property=MemoryHigh=<YOUR_MEMORYHIGH> \
> >         --property=OOMPolicy=kill \
> >         --unit="bitbake-run" \
> >         bitbake [...]
> >
> > to somewhat limit the resources, see systemd.resource-control(5) and
> > systemd.scope(5) for details regarding the used parameters.
> >
> > Best regards,
> > Ulrich
>
>
> Hi Ulrich,
>
> Using systemd-run is something that I've been meaning to try so it's good to
> hear that you're doing it.
>
> Do you have any experiences to share about this approach ?

we currently use the systemd-run approach mainly to make resource
sharing on our build servers a bit fairer between colleagues when
building customer BSPs.

One nice side effect is that every single process of the control group
gets terminated when the systemd-run scope is terminated. No stray
processes are left behind by mistake.

> Do you do world builds?

So far, I am not using it for world builds. Our use case is more about
encapsulating individual build jobs and avoiding that a single build
grabs too much of the machine, rather than stress-testing the setup with
"bitbake world".

> Do you run more than one systemd-run build on a given machine?

Yes, in practice we do end up with multiple builds running in parallel
on the same machine, since (hopefully ;) all colleagues use the same
systemd-run approach.

> Of course it doesn't prevent bitbake from pushing the cgroup contained system
> harder that it can handle and bitbake doesn't have any feedback so that's
> something
> to consider when doing the jobserver or free memory build regulation work.

Right. For our purpose, though, it works reasonably well as a pragmatic
way to improve sharing/fairness on multi-user build machines.

Best regards,
Ulrich


> ../Randy
>
>
> >
> >
> >> Could these three variables achieve this?
> >>
> >>   BB_PRESSURE_MAX_CPU, BB_PRESSURE_MAX_IO , BB_PRESSURE_MAX_MEMORY
> >>
> >> I see the range for these variables is from 1 to 1000000, so I would
> >> like to understand if there is a way to specify their values as a
> >> function of the available RAM.
> >>
> >> Regards,
> >>
> >> Luis
> >>
> >>
> >> This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential
> >> information that may be proprietary, privileged, or copyrighted under applicable law. If you are not the intended
> >> recipient, do not read, copy, or forward this email message or any attachments, delete this email message and any
> >> attachments immediately.
> >>
> >>
> >>
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >> Links: You receive all messages sent to this group.
> >> View/Reply Online (#66645):https://lists.yoctoproject.org/g/yocto/message/66645
> >> Mute This Topic:https://lists.yoctoproject.org/mt/120272854/3616765
> >> Group Owner:yocto+owner@lists.yoctoproject.org
> >> Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub [randy.macleod@windriver.com]
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >>
>
> --
> # Randy MacLeod
> # Wind River Linux
-- 
Pengutronix e.K.                           | Ulrich Ölmann               |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [yocto] Limiting Yocto Memory Usage

[Ferry Toth]

Hi,

Op 14-07-2026 om 23:33 schreef Randy MacLeod:
> Hi Luis,
> 
> On 2026-07-10 17:26, Luis Merayo via lists.yoctoproject.org wrote:
>> Hi,
>>
>> I would like to know if there is a way to limit Bitbake's Memory usage 
>> to a specific percentage of the available RAM. For example, can the 
>> Yocto build be limited to use a maximum of 70% of RAM?
> Not without using a container to do the build.
>>
>> Could these three variables achieve this?
>>
>> BB_PRESSURE_MAX_CPU <https://docs.yoctoproject.org/dev/ref-manual/ 
>> variables.html#term-BB_PRESSURE_MAX_CPU>, BB_PRESSURE_MAX_IO <https:// 
>> docs.yoctoproject.org/dev/ref-manual/variables.html#term- 
>> BB_PRESSURE_MAX_IO> , BB_PRESSURE_MAX_MEMORY <https:// 
>> docs.yoctoproject.org/dev/ref-manual/variables.html#term- 
>> BB_PRESSURE_MAX_MEMORY>
>>
>> I see the range for these variables is from 1 to 1000000, so I would 
>> like to understand if there is a way to specify their values as a 
>> function of the available RAM.
> No, there is only pressure when resources are over-commited so if you 
> have free memory, there's no over-commit.
> https://docs.kernel.org/accounting/psi.html
> 
> That's the design intent of the bitbake PRESSURE regulation system.
> 
> There are plans to work on additional build system regulation.
> 
> The first idea is to use the jobserver design:
> https://lore.kernel.org/openembedded-core/?q=GNU+AND+jobserver+ 
> +AND+f%3Amacleod
> and that would indirectly limit memory consumption since there would be 
> fewer instances of gcc / rustc / ... running
> at once.

I do that, since 2 years or so. 
https://github.com/htot/meta-intel-edison/commits/whinlatter/

In principle you need to catch the call to make and ninja (to force them 
to use the jobserver) and patch bitbake (see setup.sh) to act as a 
jobserver.

Now got it working on wrynose, still need to push that. Maybe later 
today.With wrynose ninja has support for jobserver so doesn't need 
patching, just a bbappend.

With this you can limit the number of compile jobs. On my builds I found 
g++ compiles are the heaviest, about 1GB RAM each. So, with 16 cores I 
need 16 GB.
Separately I found linking nodejs takes 5 x 5GB (5 jobs linking with 
LTO?). So, I limited that to just one (since both nodejs and 
nodejs-native can be built at the same time).

I'm not building rust so there may be other bottlenecks to resolve.

> There's also an even less clear idea about being able to restrict the 
> build by specifying memory usage.
> That shouldn't be too hard to implement in bitbake in the same point 
> where the pressure checks happen.
> Are you interested in working on that ?
> 
> There are of course other levers to pull to reduce CPU/memory consumption:
> https://docs.yoctoproject.org/dev-manual/limiting-resources.html
> 
> Have you tried them ?
> 
> 
>>
>> Regards,
>>
>> Luis
>>
>>
>>
>> File:RidgeRun.ai banner.png <https://ridgerun.ai/>
>> This email and any attachments are intended for the sole use of the 
>> named recipient(s) and contain(s) confidential information that may be 
>> proprietary, privileged, or copyrighted under applicable law. If you 
>> are not the intended recipient, do not read, copy, or forward this 
>> email message or any attachments, delete this email message and any 
>> attachments immediately.
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#66637):https://lists.yoctoproject.org/g/yocto/message/66637
>> Mute This Topic:https://lists.yoctoproject.org/mt/120213825/3616765
>> Group Owner:yocto+owner@lists.yoctoproject.org
>> Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub [randy.macleod@windriver.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
> 
> -- 
> # Randy MacLeod
> # Wind River Linux
> 

Re: [yocto] Limiting Yocto Memory Usage

[Ferry Toth]

Hi

Op 28-07-2026 om 09:30 schreef Ferry Toth:
> Hi,
> 
> Op 14-07-2026 om 23:33 schreef Randy MacLeod:
>> Hi Luis,
>>
>> On 2026-07-10 17:26, Luis Merayo via lists.yoctoproject.org wrote:
>>> Hi,
>>>
>>> I would like to know if there is a way to limit Bitbake's Memory 
>>> usage to a specific percentage of the available RAM. For example, can 
>>> the Yocto build be limited to use a maximum of 70% of RAM?
>> Not without using a container to do the build.
>>>
>>> Could these three variables achieve this?
>>>
>>> BB_PRESSURE_MAX_CPU <https://docs.yoctoproject.org/dev/ref-manual/ 
>>> variables.html#term-BB_PRESSURE_MAX_CPU>, BB_PRESSURE_MAX_IO 
>>> <https:// docs.yoctoproject.org/dev/ref-manual/variables.html#term- 
>>> BB_PRESSURE_MAX_IO> , BB_PRESSURE_MAX_MEMORY <https:// 
>>> docs.yoctoproject.org/dev/ref-manual/variables.html#term- 
>>> BB_PRESSURE_MAX_MEMORY>
>>>
>>> I see the range for these variables is from 1 to 1000000, so I would 
>>> like to understand if there is a way to specify their values as a 
>>> function of the available RAM.
>> No, there is only pressure when resources are over-commited so if you 
>> have free memory, there's no over-commit.
>> https://docs.kernel.org/accounting/psi.html
>>
>> That's the design intent of the bitbake PRESSURE regulation system.
>>
>> There are plans to work on additional build system regulation.
>>
>> The first idea is to use the jobserver design:
>> https://lore.kernel.org/openembedded-core/?q=GNU+AND+jobserver+ 
>> +AND+f%3Amacleod
>> and that would indirectly limit memory consumption since there would 
>> be fewer instances of gcc / rustc / ... running
>> at once.
> 
> I do that, since 2 years or so. https://github.com/htot/meta-intel- 
> edison/commits/whinlatter/
> 
> In principle you need to catch the call to make and ninja (to force them 
> to use the jobserver) and patch bitbake (see setup.sh) to act as a 
> jobserver.
> 
> Now got it working on wrynose, still need to push that. Maybe later 
> today.With wrynose ninja has support for jobserver so doesn't need 
> patching, just a bbappend.

Pushed my wrynose branch now: 
https://github.com/htot/meta-intel-edison/tree/wrynose

> With this you can limit the number of compile jobs. On my builds I found 
> g++ compiles are the heaviest, about 1GB RAM each. So, with 16 cores I 
> need 16 GB.
> Separately I found linking nodejs takes 5 x 5GB (5 jobs linking with 
> LTO?). So, I limited that to just one (since both nodejs and nodejs- 
> native can be built at the same time).
> 
> I'm not building rust so there may be other bottlenecks to resolve.
> 
>> There's also an even less clear idea about being able to restrict the 
>> build by specifying memory usage.
>> That shouldn't be too hard to implement in bitbake in the same point 
>> where the pressure checks happen.
>> Are you interested in working on that ?
>>
>> There are of course other levers to pull to reduce CPU/memory 
>> consumption:
>> https://docs.yoctoproject.org/dev-manual/limiting-resources.html
>>
>> Have you tried them ?
>>
>>
>>>
>>> Regards,
>>>
>>> Luis
>>>
>>>
>>>
>>> File:RidgeRun.ai banner.png <https://ridgerun.ai/>
>>> This email and any attachments are intended for the sole use of the 
>>> named recipient(s) and contain(s) confidential information that may 
>>> be proprietary, privileged, or copyrighted under applicable law. If 
>>> you are not the intended recipient, do not read, copy, or forward 
>>> this email message or any attachments, delete this email message and 
>>> any attachments immediately.
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#66637):https://lists.yoctoproject.org/g/yocto/ 
>>> message/66637
>>> Mute This Topic:https://lists.yoctoproject.org/mt/120213825/3616765
>>> Group Owner:yocto+owner@lists.yoctoproject.org
>>> Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub 
>>> [randy.macleod@windriver.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>>
>> -- 
>> # Randy MacLeod
>> # Wind River Linux
>>
> 
> 

Re: [yocto] Limiting Yocto Memory Usage

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 4607 bytes --]

On 2026-07-28 16:35, Ferry Toth via lists.yoctoproject.org wrote:
> Hi
>
> Op 28-07-2026 om 09:30 schreef Ferry Toth:
>> Hi,
>>
>> Op 14-07-2026 om 23:33 schreef Randy MacLeod:
>>> Hi Luis,
>>>
>>> On 2026-07-10 17:26, Luis Merayo via lists.yoctoproject.org wrote:
>>>> Hi,
>>>>
>>>> I would like to know if there is a way to limit Bitbake's Memory 
>>>> usage to a specific percentage of the available RAM. For example, 
>>>> can the Yocto build be limited to use a maximum of 70% of RAM?
>>> Not without using a container to do the build.
>>>>
>>>> Could these three variables achieve this?
>>>>
>>>> BB_PRESSURE_MAX_CPU <https://docs.yoctoproject.org/dev/ref-manual/ 
>>>> variables.html#term-BB_PRESSURE_MAX_CPU>, BB_PRESSURE_MAX_IO 
>>>> <https:// docs.yoctoproject.org/dev/ref-manual/variables.html#term- 
>>>> BB_PRESSURE_MAX_IO> , BB_PRESSURE_MAX_MEMORY <https:// 
>>>> docs.yoctoproject.org/dev/ref-manual/variables.html#term- 
>>>> BB_PRESSURE_MAX_MEMORY>
>>>>
>>>> I see the range for these variables is from 1 to 1000000, so I 
>>>> would like to understand if there is a way to specify their values 
>>>> as a function of the available RAM.
>>> No, there is only pressure when resources are over-commited so if 
>>> you have free memory, there's no over-commit.
>>> https://docs.kernel.org/accounting/psi.html
>>>
>>> That's the design intent of the bitbake PRESSURE regulation system.
>>>
>>> There are plans to work on additional build system regulation.
>>>
>>> The first idea is to use the jobserver design:
>>> https://lore.kernel.org/openembedded-core/?q=GNU+AND+jobserver+ 
>>> +AND+f%3Amacleod
>>> and that would indirectly limit memory consumption since there would 
>>> be fewer instances of gcc / rustc / ... running
>>> at once.
>>
>> I do that, since 2 years or so. https://github.com/htot/meta-intel- 
>> edison/commits/whinlatter/
>>
>> In principle you need to catch the call to make and ninja (to force 
>> them to use the jobserver) and patch bitbake (see setup.sh) to act as 
>> a jobserver.
>>
>> Now got it working on wrynose, still need to push that. Maybe later 
>> today.With wrynose ninja has support for jobserver so doesn't need 
>> patching, just a bbappend.
>
> Pushed my wrynose branch now: 
> https://github.com/htot/meta-intel-edison/tree/wrynose

Nice, are you interested in trying to get that or something based on 
that work merged into oe-core/master ?
I have been wanting to do for a while but other responsibilities keep 
pushing it to the back burner or out of the kitchen completely !

../Randy


>
>> With this you can limit the number of compile jobs. On my builds I 
>> found g++ compiles are the heaviest, about 1GB RAM each. So, with 16 
>> cores I need 16 GB.
>> Separately I found linking nodejs takes 5 x 5GB (5 jobs linking with 
>> LTO?). So, I limited that to just one (since both nodejs and nodejs- 
>> native can be built at the same time).
>>
>> I'm not building rust so there may be other bottlenecks to resolve.
>>
>>> There's also an even less clear idea about being able to restrict 
>>> the build by specifying memory usage.
>>> That shouldn't be too hard to implement in bitbake in the same point 
>>> where the pressure checks happen.
>>> Are you interested in working on that ?
>>>
>>> There are of course other levers to pull to reduce CPU/memory 
>>> consumption:
>>> https://docs.yoctoproject.org/dev-manual/limiting-resources.html
>>>
>>> Have you tried them ?
>>>
>>>
>>>>
>>>> Regards,
>>>>
>>>> Luis
>>>>
>>>>
>>>>
>>>> File:RidgeRun.ai banner.png <https://ridgerun.ai/>
>>>> This email and any attachments are intended for the sole use of the 
>>>> named recipient(s) and contain(s) confidential information that may 
>>>> be proprietary, privileged, or copyrighted under applicable law. If 
>>>> you are not the intended recipient, do not read, copy, or forward 
>>>> this email message or any attachments, delete this email message 
>>>> and any attachments immediately.
>>>>
>>>>
>>>>
>>>
>>> -- 
>>> # Randy MacLeod
>>> # Wind River Linux
>>>
>>
>>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#66668):https://lists.yoctoproject.org/g/yocto/message/66668
> Mute This Topic:https://lists.yoctoproject.org/mt/120272854/3616765
> Group Owner:yocto+owner@lists.yoctoproject.org
> Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 8362 bytes --]

Re: [yocto] Limiting Yocto Memory Usage

[Ferry Toth]

Hi,

Op 28-07-2026 om 23:20 schreef Randy MacLeod:
> On 2026-07-28 16:35, Ferry Toth via lists.yoctoproject.org wrote:
>> Hi
>>
>> Op 28-07-2026 om 09:30 schreef Ferry Toth:
>>> Hi,
>>>
>>> Op 14-07-2026 om 23:33 schreef Randy MacLeod:
>>>> Hi Luis,
>>>>
>>>> On 2026-07-10 17:26, Luis Merayo via lists.yoctoproject.org wrote:
>>>>> Hi,
>>>>>
>>>>> I would like to know if there is a way to limit Bitbake's Memory 
>>>>> usage to a specific percentage of the available RAM. For example, 
>>>>> can the Yocto build be limited to use a maximum of 70% of RAM?
>>>> Not without using a container to do the build.
>>>>>
>>>>> Could these three variables achieve this?
>>>>>
>>>>> BB_PRESSURE_MAX_CPU <https://docs.yoctoproject.org/dev/ref-manual/ 
>>>>> variables.html#term-BB_PRESSURE_MAX_CPU>, BB_PRESSURE_MAX_IO 
>>>>> <https:// 
>>>>> docs.yoctoproject.org/dev/ref-manual/variables.html#term- 
>>>>> BB_PRESSURE_MAX_IO> , BB_PRESSURE_MAX_MEMORY <https:// 
>>>>> docs.yoctoproject.org/dev/ref-manual/variables.html#term- 
>>>>> BB_PRESSURE_MAX_MEMORY>
>>>>>
>>>>> I see the range for these variables is from 1 to 1000000, so I 
>>>>> would like to understand if there is a way to specify their values 
>>>>> as a function of the available RAM.
>>>> No, there is only pressure when resources are over-commited so if 
>>>> you have free memory, there's no over-commit.
>>>> https://docs.kernel.org/accounting/psi.html
>>>>
>>>> That's the design intent of the bitbake PRESSURE regulation system.
>>>>
>>>> There are plans to work on additional build system regulation.
>>>>
>>>> The first idea is to use the jobserver design:
>>>> https://lore.kernel.org/openembedded-core/?q=GNU+AND+jobserver+ 
>>>> +AND+f%3Amacleod
>>>> and that would indirectly limit memory consumption since there 
>>>> would be fewer instances of gcc / rustc / ... running
>>>> at once.
>>>
>>> I do that, since 2 years or so. https://github.com/htot/meta-intel- 
>>> edison/commits/whinlatter/
>>>
>>> In principle you need to catch the call to make and ninja (to force 
>>> them to use the jobserver) and patch bitbake (see setup.sh) to act 
>>> as a jobserver.
>>>
>>> Now got it working on wrynose, still need to push that. Maybe later 
>>> today.With wrynose ninja has support for jobserver so doesn't need 
>>> patching, just a bbappend.
>>
>> Pushed my wrynose branch now: 
>> https://github.com/htot/meta-intel-edison/tree/wrynose
>
> Nice, are you interested in trying to get that or something based on 
> that work merged into oe-core/master ?
> I have been wanting to do for a while but other responsibilities keep 
> pushing it to the back burner or out of the kitchen completely !
>
Yeah that would be great. The patches were originally suggested by 
Richard Purdie. What spooks me a bit (based on earlier experience) is 
what additional stuff (self tests?) would be needed to get this accepted.
>
> ../Randy
>
>
>>
>>> With this you can limit the number of compile jobs. On my builds I 
>>> found g++ compiles are the heaviest, about 1GB RAM each. So, with 16 
>>> cores I need 16 GB.
>>> Separately I found linking nodejs takes 5 x 5GB (5 jobs linking with 
>>> LTO?). So, I limited that to just one (since both nodejs and nodejs- 
>>> native can be built at the same time).
>>>
>>> I'm not building rust so there may be other bottlenecks to resolve.
>>>
>>>> There's also an even less clear idea about being able to restrict 
>>>> the build by specifying memory usage.
>>>> That shouldn't be too hard to implement in bitbake in the same 
>>>> point where the pressure checks happen.
>>>> Are you interested in working on that ?
>>>>
>>>> There are of course other levers to pull to reduce CPU/memory 
>>>> consumption:
>>>> https://docs.yoctoproject.org/dev-manual/limiting-resources.html
>>>>
>>>> Have you tried them ?
>>>>
>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Luis
>>>>>
>>>>>
>>>>>
>>>>> File:RidgeRun.ai banner.png <https://ridgerun.ai/>
>>>>> This email and any attachments are intended for the sole use of 
>>>>> the named recipient(s) and contain(s) confidential information 
>>>>> that may be proprietary, privileged, or copyrighted under 
>>>>> applicable law. If you are not the intended recipient, do not 
>>>>> read, copy, or forward this email message or any attachments, 
>>>>> delete this email message and any attachments immediately.
>>>>>
>>>>>
>>>>>
>>>>
>>>> -- 
>>>> # Randy MacLeod
>>>> # Wind River Linux
>>>>
>>>
>>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#66668):https://lists.yoctoproject.org/g/yocto/message/66668
>> Mute This Topic:https://lists.yoctoproject.org/mt/120272854/3616765
>> Group Owner:yocto+owner@lists.yoctoproject.org
>> Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub [randy.macleod@windriver.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>
> -- 
> # Randy MacLeod
> # Wind River Linux

Re: [yocto] Limiting Yocto Memory Usage

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 1042 bytes --]

On 2026-07-28 17:36, Ferry Toth wrote:
>> Nice, are you interested in trying to get that or something based on 
>> that work merged into oe-core/master ?
>> I have been wanting to do for a while but other responsibilities keep 
>> pushing it to the back burner or out of the kitchen completely !
>>
> Yeah that would be great. The patches were originally suggested by 
> Richard Purdie. What spooks me a bit (based on earlier experience) is 
> what additional stuff (self tests?) would be needed to get this accepted.

I hear what you're saying.
I worked on Martin's jobserver patch set for a while and I was trying to
add some tests to prove that it was working well.

I think there's also some questions about the set of features that we 
would implement.

I've send the set of patchs to you at this email address. Look at them 
if you like.
I've been too busy to review how they differ from your approach.

All for now, time for another freaking meeting!  ;-)

../Randy

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 1717 bytes --]