Re: Raspberry Pi 3B energy consumption

Re: Raspberry Pi 3B energy consumption

[Alex Bennée]

Polzini <laurentpolzin@gmail.com> writes:

> Dear Sir/Madam, My name is Clément Aldebert, and I am a third-year computer science student at Paul Sabatier University.
> Together with my teammate Laurent Polzin, we are working on a project supervised by Millian Poquet. Our project focuses on
> reducing the energy consumption of QEMU’s Raspberry Pi 3B emulation, as we have observed that power usage is quite high
> even when the system is idle.

Can you qualify that a bit more? What are you running the emulation on?
What power usage do you see?

Power consumption of the emulation is really just down to computational
efficiency of the emulation and if we ever get to a point that QEMU can
go idle giving the host system a chance to reach a lower power state.

> Ultimately, we also aim to enable emulation on a single CPU core.

If you want to run all emulation on a single core you could use -icount
to do that. It is less efficient though.

> We are reaching out to
> inquire if you have any advice, resources, or suggestions for implementing power management in QEMU, particularly for the
> Raspberry Pi 3B model. Additionally, we would like to understand any constraints or best practices regarding adding new
> code or features to QEMU.

What documentation that exists can be found in the developers section of
the manual:

  https://qemu.readthedocs.io/en/master/devel/index.html

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: Raspberry Pi 3B energy consumption

[clement.aldebert]

Dear Alex Bennée,

Thank you for your response and for the provided documentation link.

To clarify, when we refer to energy consumption, we are specifically looking at CPU core utilization. Our goal is to ensure that when we turn off CPU cores on the emulated Raspberry Pi 3B, the host machine does not keep its corresponding cores running at 100% utilization.

This is particularly relevant for our use case, as our supervisor, M. Poquet, intends to use this project for teaching operating system development. Students using QEMU to emulate a Raspberry Pi 3B may have different host machines, so our objective is to implement a general approach that ensures CPU core deactivation works properly regardless of the host system.

So far, our measurements indicate that even after turning off cores in the emulated system, all CPU cores on the host remain at 100% utilization. We are exploring ways to improve this behavior.

We have looked into using -icount, but we would like to find a more viable solution, particularly for the Cortex-A53 CPU used in the Raspberry Pi 3B. Ideally, we would like to be able to use -smp 1 without being constrained by QEMU’s limitations and to implement proper CPU core shutdown, similar to what is available on x86-64 architectures.

Would you have any recommendations or insights on achieving better power management in this scenario?

Best regards,
Clément Aldebert & Laurent Polzin

----- Mail original -----
De: "Alex Bennée" <alex.bennee@linaro.org>
À: "Polzini" <laurentpolzin@gmail.com>
Cc: qemu-devel@nongnu.org, qemu-arm@nongnu.org, "millian poquet" <millian.poquet@univ-tlse3.fr>, "clement aldebert" <clement.aldebert@univ-tlse3.fr>
Envoyé: Mercredi 19 Mars 2025 16:57:41
Objet: Re: Raspberry Pi 3B energy consumption

Polzini <laurentpolzin@gmail.com> writes:

> Dear Sir/Madam, My name is Clément Aldebert, and I am a third-year computer science student at Paul Sabatier University.
> Together with my teammate Laurent Polzin, we are working on a project supervised by Millian Poquet. Our project focuses on
> reducing the energy consumption of QEMU’s Raspberry Pi 3B emulation, as we have observed that power usage is quite high
> even when the system is idle.

Can you qualify that a bit more? What are you running the emulation on?
What power usage do you see?

Power consumption of the emulation is really just down to computational
efficiency of the emulation and if we ever get to a point that QEMU can
go idle giving the host system a chance to reach a lower power state.

> Ultimately, we also aim to enable emulation on a single CPU core.

If you want to run all emulation on a single core you could use -icount
to do that. It is less efficient though.

> We are reaching out to
> inquire if you have any advice, resources, or suggestions for implementing power management in QEMU, particularly for the
> Raspberry Pi 3B model. Additionally, we would like to understand any constraints or best practices regarding adding new
> code or features to QEMU.

What documentation that exists can be found in the developers section of
the manual:

  https://qemu.readthedocs.io/en/master/devel/index.html

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: Raspberry Pi 3B energy consumption

[Peter Maydell]

On Thu, 20 Mar 2025 at 19:30, <clement.aldebert@univ-tlse3.fr> wrote:
> To clarify, when we refer to energy consumption, we are specifically looking at CPU core utilization. Our goal is to ensure that when we turn off CPU cores on the emulated Raspberry Pi 3B, the host machine does not keep its corresponding cores running at 100% utilization.
>
> This is particularly relevant for our use case, as our supervisor, M. Poquet, intends to use this project for teaching operating system development. Students using QEMU to emulate a Raspberry Pi 3B may have different host machines, so our objective is to implement a general approach that ensures CPU core deactivation works properly regardless of the host system.
>
> So far, our measurements indicate that even after turning off cores in the emulated system, all CPU cores on the host remain at 100% utilization. We are exploring ways to improve this behavior.
>
> We have looked into using -icount, but we would like to find a more viable solution, particularly for the Cortex-A53 CPU used in the Raspberry Pi 3B. Ideally, we would like to be able to use -smp 1 without being constrained by QEMU’s limitations and to implement proper CPU core shutdown, similar to what is available on x86-64 architectures.
>
> Would you have any recommendations or insights on achieving better power management in this scenario?

You should check what mechanism you're trying to use to "turn off the cores";
there are several possibilities, some of which are "this ought to
work, so if it isn't doing the right thing that's a bug to investigate",
and some of which are "this is a feature that's not implemented at
all right now":

In the "should work" category:
 * if the CPU executes a WFI instruction we will stop executing code
   (until the next interrupt)
 * If you are using QEMU's PSCI implementation and you power the CPU
   down with e.g. PSCI CPU_OFF we should also stop executing code.
   (NB this is different from if you're running real guest firmware and
   the firmware is handling PSCI calls)

In the "not implemented" category:
 * If you are trying to stop the CPU with a WFE instruction, we do not
   stop executing code (we emulate it as a NOP, which is architecturally
   permitted).
 * If you are trying to stop the CPU with some raspi-SoC-specific power
   controller, we don't currently have a model of that, I think

If you need a working raspi SoC power controller device, the answer
is that that power controller needs to be modelled so that when the
guest uses it to turn off a CPU it makes the appropriate calls to
arm_set_cpu_off() etc.  This should be straightforward assuming that
you have the appropriate datasheets/specs/etc describing the hardware
behaviour.

If you need WFE to work, that's certainly feasible and something it would
be nice to see, but potentially quite a bit of work in the guts of QEMU's
arm emulation. (Basically going to sleep on WFE is easy but then making
sure that all the events  and situations that need to wake up a WFE is
tedious. We implement sleep-on-WFI but not sleep-on-WFI because the set
of WFI-wakeup events is rather smaller than the WFE-wakeup events.) It's
been in the "we really should implement this but since the only downside
is the host CPUs spinning, we've never got round to it" bucket for years.

(In the cases where we do stop executing code that means that that
vCPU thread should no longer be doing a lot of work; obviously
if your system has more vCPUs than host CPUs then the host CPUs
will still all be busy.)

Your other option here is "stop trying to use the raspberry pi
machine types". The "virt" board, because it doesn't need to try
to implement the same behaviour as an exact machine type, has a
much easier job, and it already supports "you can have just one
CPU with -smp 1" and "you can have an SMP system where you power
off some of the CPUs while running". Basically "virt" is actively
maintained, whereas the raspi machines are in the "Odd Fixes" state,
which means we'll try to review patches but nobody is actively
working on trying to make them better.

thanks
-- PMM

Re: Raspberry Pi 3B energy consumption

[Peter Maydell]

On Thu, 20 Mar 2025 at 20:09, Peter Maydell <peter.maydell@linaro.org> wrote:
> If you need WFE to work, that's certainly feasible and something it would
> be nice to see, but potentially quite a bit of work in the guts of QEMU's
> arm emulation. (Basically going to sleep on WFE is easy but then making
> sure that all the events  and situations that need to wake up a WFE is
> tedious. We implement sleep-on-WFI but not sleep-on-WFI because the set

should read "sleep-on-WFI but not sleep-on-WFE", of course. Oops...

> of WFI-wakeup events is rather smaller than the WFE-wakeup events.) It's
> been in the "we really should implement this but since the only downside
> is the host CPUs spinning, we've never got round to it" bucket for years.

-- PMM

Re: Raspberry Pi 3B energy consumption

[clement.aldebert]

Dear Peter Maydell,

Thank you for your detailed response.

We will take a closer look at why, when using WFI, the affected cores still appear to be at 100% utilization in htop. Additionally, we will investigate whether implementing WFE in QEMU would be necessary to achieve proper CPU core shutdown.

Currently, we are not using PSCI, but this does seem like a promising approach that we will explore further.

Regarding the use of the "virt" machine, our supervisor, M. Poquet, requires the Raspberry Pi 3B specifically for his courses due to its hardware specifications. Therefore, switching to "virt" would only be considered as a last resort.

Thank you again for your insights. Any further recommendations would be greatly appreciated.

Best regards,
Clément Aldebert & Laurent Polzin



----- Mail original -----
De: "Peter Maydell" <peter.maydell@linaro.org>
À: "clement aldebert" <clement.aldebert@univ-tlse3.fr>
Cc: "Alex Bennée" <alex.bennee@linaro.org>, "qemu-devel" <qemu-devel@nongnu.org>, "millian.poquet" <millian.poquet@univ-tlse3.fr>, "laurent polzin" <laurent.polzin@univ-tlse3.fr>
Envoyé: Jeudi 20 Mars 2025 21:11:52
Objet: Re: Raspberry Pi 3B energy consumption

On Thu, 20 Mar 2025 at 20:09, Peter Maydell <peter.maydell@linaro.org> wrote:
> If you need WFE to work, that's certainly feasible and something it would
> be nice to see, but potentially quite a bit of work in the guts of QEMU's
> arm emulation. (Basically going to sleep on WFE is easy but then making
> sure that all the events  and situations that need to wake up a WFE is
> tedious. We implement sleep-on-WFI but not sleep-on-WFI because the set

should read "sleep-on-WFI but not sleep-on-WFE", of course. Oops...

> of WFI-wakeup events is rather smaller than the WFE-wakeup events.) It's
> been in the "we really should implement this but since the only downside
> is the host CPUs spinning, we've never got round to it" bucket for years.

-- PMM