[PATCH] runqueue: show the current pressure as debug messages

[PATCH] runqueue: show the current pressure as debug messages

[Antonin Godard]

The current pressure logic only prints pressure information when the
limits set by BB_PRESSURE_* variables are exceeded. But in order to
determine the best pressure limits for a given host, it can be useful to
monitor pressure information after setting BB_PRESSURE_* to the maximum
(1000000) and get a sense of how high the pressure can be.

Add three bb.debug() calls showing that information.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 lib/bb/runqueue.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index 80f3d3282f..cae4bfa3e5 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -206,6 +206,9 @@ class RunQueueScheduler(object):
                 cpu_pressure = (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) / tdiff
                 io_pressure = (float(curr_io_pressure) - float(self.prev_io_pressure)) / tdiff
                 memory_pressure = (float(curr_memory_pressure) - float(self.prev_memory_pressure)) / tdiff
+                bb.debug(3, f"Current CPU pressure: {cpu_pressure}")
+                bb.debug(3, f"Current IO pressure: {io_pressure}")
+                bb.debug(3, f"Current Memory pressure: {memory_pressure}")
                 exceeds_cpu_pressure =  self.rq.max_cpu_pressure and cpu_pressure > self.rq.max_cpu_pressure
                 exceeds_io_pressure =  self.rq.max_io_pressure and io_pressure > self.rq.max_io_pressure
                 exceeds_memory_pressure =  self.rq.max_memory_pressure and memory_pressure > self.rq.max_memory_pressure

---
base-commit: f68b513c38fa33c89236efbaab2674a25983d5e1
change-id: 20250701-pressure-messages-b47c882bca4a

Best regards,
--  
Antonin Godard <antonin.godard@bootlin.com>

Re: [bitbake-devel] [PATCH] runqueue: show the current pressure as debug messages

[Richard Purdie]

On Tue, 2025-07-01 at 11:53 +0200, Antonin Godard via lists.openembedded.org wrote:
> The current pressure logic only prints pressure information when the
> limits set by BB_PRESSURE_* variables are exceeded. But in order to
> determine the best pressure limits for a given host, it can be useful to
> monitor pressure information after setting BB_PRESSURE_* to the maximum
> (1000000) and get a sense of how high the pressure can be.
> 
> Add three bb.debug() calls showing that information.
> 
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
>  lib/bb/runqueue.py | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
> index 80f3d3282f..cae4bfa3e5 100644
> --- a/lib/bb/runqueue.py
> +++ b/lib/bb/runqueue.py
> @@ -206,6 +206,9 @@ class RunQueueScheduler(object):
>                  cpu_pressure = (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) / tdiff
>                  io_pressure = (float(curr_io_pressure) - float(self.prev_io_pressure)) / tdiff
>                  memory_pressure = (float(curr_memory_pressure) - float(self.prev_memory_pressure)) / tdiff
> +                bb.debug(3, f"Current CPU pressure: {cpu_pressure}")
> +                bb.debug(3, f"Current IO pressure: {io_pressure}")
> +                bb.debug(3, f"Current Memory pressure: {memory_pressure}")
>                  exceeds_cpu_pressure =  self.rq.max_cpu_pressure and cpu_pressure > self.rq.max_cpu_pressure
>                  exceeds_io_pressure =  self.rq.max_io_pressure and io_pressure > self.rq.max_io_pressure
>                  exceeds_memory_pressure =  self.rq.max_memory_pressure and memory_pressure > self.rq.max_memory_pressure
> 

I'm torn on this, debug messages aren't without their overhead even if
not printed. How often do these end up outputted? If nothing else there
should be one bb.debug call and not 3...

Cheers,

Richard

Re: [bitbake-devel] [PATCH] runqueue: show the current pressure as debug messages

[Antonin Godard]

On Tue Jul 1, 2025 at 11:23 PM CEST, Richard Purdie wrote:
> On Tue, 2025-07-01 at 11:53 +0200, Antonin Godard via lists.openembedded.org wrote:
>> The current pressure logic only prints pressure information when the
>> limits set by BB_PRESSURE_* variables are exceeded. But in order to
>> determine the best pressure limits for a given host, it can be useful to
>> monitor pressure information after setting BB_PRESSURE_* to the maximum
>> (1000000) and get a sense of how high the pressure can be.
>> 
>> Add three bb.debug() calls showing that information.
>> 
>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>> ---
>>  lib/bb/runqueue.py | 3 +++
>>  1 file changed, 3 insertions(+)
>> 
>> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
>> index 80f3d3282f..cae4bfa3e5 100644
>> --- a/lib/bb/runqueue.py
>> +++ b/lib/bb/runqueue.py
>> @@ -206,6 +206,9 @@ class RunQueueScheduler(object):
>>                  cpu_pressure = (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) / tdiff
>>                  io_pressure = (float(curr_io_pressure) - float(self.prev_io_pressure)) / tdiff
>>                  memory_pressure = (float(curr_memory_pressure) - float(self.prev_memory_pressure)) / tdiff
>> +                bb.debug(3, f"Current CPU pressure: {cpu_pressure}")
>> +                bb.debug(3, f"Current IO pressure: {io_pressure}")
>> +                bb.debug(3, f"Current Memory pressure: {memory_pressure}")
>>                  exceeds_cpu_pressure =  self.rq.max_cpu_pressure and cpu_pressure > self.rq.max_cpu_pressure
>>                  exceeds_io_pressure =  self.rq.max_io_pressure and io_pressure > self.rq.max_io_pressure
>>                  exceeds_memory_pressure =  self.rq.max_memory_pressure and memory_pressure > self.rq.max_memory_pressure
>> 
>
> I'm torn on this, debug messages aren't without their overhead even if
> not printed. How often do these end up outputted? If nothing else there
> should be one bb.debug call and not 3...

Actually, quite a lot, so I should reduce this to one call, or find out a way to
reduce the amount of messages printed.

Thanks,
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [bitbake-devel] [PATCH] runqueue: show the current pressure as debug messages

[Quentin Schulz]

Hi Antonin,

On 7/2/25 8:21 AM, Antonin Godard via lists.openembedded.org wrote:
> On Tue Jul 1, 2025 at 11:23 PM CEST, Richard Purdie wrote:
>> On Tue, 2025-07-01 at 11:53 +0200, Antonin Godard via lists.openembedded.org wrote:
>>> The current pressure logic only prints pressure information when the
>>> limits set by BB_PRESSURE_* variables are exceeded. But in order to
>>> determine the best pressure limits for a given host, it can be useful to
>>> monitor pressure information after setting BB_PRESSURE_* to the maximum
>>> (1000000) and get a sense of how high the pressure can be.
>>>
>>> Add three bb.debug() calls showing that information.
>>>
>>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>>> ---
>>>   lib/bb/runqueue.py | 3 +++
>>>   1 file changed, 3 insertions(+)
>>>
>>> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
>>> index 80f3d3282f..cae4bfa3e5 100644
>>> --- a/lib/bb/runqueue.py
>>> +++ b/lib/bb/runqueue.py
>>> @@ -206,6 +206,9 @@ class RunQueueScheduler(object):
>>>                   cpu_pressure = (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) / tdiff
>>>                   io_pressure = (float(curr_io_pressure) - float(self.prev_io_pressure)) / tdiff
>>>                   memory_pressure = (float(curr_memory_pressure) - float(self.prev_memory_pressure)) / tdiff
>>> +                bb.debug(3, f"Current CPU pressure: {cpu_pressure}")
>>> +                bb.debug(3, f"Current IO pressure: {io_pressure}")
>>> +                bb.debug(3, f"Current Memory pressure: {memory_pressure}")
>>>                   exceeds_cpu_pressure =  self.rq.max_cpu_pressure and cpu_pressure > self.rq.max_cpu_pressure
>>>                   exceeds_io_pressure =  self.rq.max_io_pressure and io_pressure > self.rq.max_io_pressure
>>>                   exceeds_memory_pressure =  self.rq.max_memory_pressure and memory_pressure > self.rq.max_memory_pressure
>>>
>>
>> I'm torn on this, debug messages aren't without their overhead even if
>> not printed. How often do these end up outputted? If nothing else there
>> should be one bb.debug call and not 3...
> 
> Actually, quite a lot, so I should reduce this to one call, or find out a way to
> reduce the amount of messages printed.
> 

We could also have this printed only every X seconds but that could make 
us miss the highest pressure before bitbake gets OOM'ed for example.

We could also only print this whenever a new max pressure is reached and 
we would only need to store the last max pressure values between rounds.

Cheers,
Quentin