[PATCH v2 0/1] timers/migration: add 'tmigr' kernel parameter to optionally disable timer migration

[PATCH v2 0/1] timers/migration: add 'tmigr' kernel parameter to optionally disable timer migration

[Sehee Jeong]

This patch introduces a kernel boot parameter to optionally disable the
timer migration.

On heterogeneous systems with big.LITTLE architectures, timer migration
may cause timers from little cores to run on big cores, or vice versa,
because core type differences are not considered in the current timer
migration logic.

In our heterogeneous system, we observed timers being migrated to big
CPU frequently, resulting in timer callbacks that could have run on
little CPUs being executed on the big CPU instead. This reduced big
CPU's idle residency and increased overall power consumption due to
higher power draw on the big CPU. Since timer_migration is a relatively
new feature, addressing the structural limitation was difficult.
Therefore, this patch adds a boot parameter to optionally disable timer
migration.

---
Changes in v2:
  - Added more description of potential issue in the commit message
  - Changed the boot parameter format to tmigr=[on|off]

Link: https://lore.kernel.org/r/20250807064849.3988-1-sehee1.jeong@samsung.com

Sehee Jeong (1):
  timers/migration: add 'tmigr' kernel parameter to optionally disable
    timer migration

 .../admin-guide/kernel-parameters.txt         |  4 ++++
 kernel/time/timer_migration.c                 | 19 +++++++++++++++++++
 2 files changed, 23 insertions(+)

-- 
2.49.0

[PATCH v2 1/1] timers/migration: add 'tmigr' kernel parameter to optionally disable timer migration

[Sehee Jeong]

On heterogeneous systems with big.LITTLE architectures, timer migration
may cause timers from little cores to run on big cores, or vice versa,
because core type differences are not considered in the current timer
migration logic. This can be undesirable in systems that require
strict power management.

For example, if timers are frequently migrated to a big CPU, it must
handle callbacks that could have run on a little CPU. This reduces the
big CPU's idle residency and increases overall energy consumption due to
higher power draw on the big CPU.

To avoid this issue, introduce an early boot parameter to optionally
disable timer migration:

    tmigr=on|off	(default: on)

When set to "off", timer migration initialization is skipped entirely.

Signed-off-by: Sehee Jeong <sehee1.jeong@samsung.com>
---
 .../admin-guide/kernel-parameters.txt         |  4 ++++
 kernel/time/timer_migration.c                 | 19 +++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index ab4c049faba9..0f5d42c046bb 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -6783,6 +6783,10 @@
 			Force threading of all interrupt handlers except those
 			marked explicitly IRQF_NO_THREAD.
 
+	tmigr		[KNL,EARLY] Enable/disable timer migration
+			Valid parameters: on, off
+			Default: on
+
 	topology=	[S390,EARLY]
 			Format: {off | on}
 			Specify if the kernel should make use of the cpu
diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c
index 72538baa7a1f..00e3740aec98 100644
--- a/kernel/time/timer_migration.c
+++ b/kernel/time/timer_migration.c
@@ -422,6 +422,8 @@ static unsigned int tmigr_crossnode_level __read_mostly;
 
 static DEFINE_PER_CPU(struct tmigr_cpu, tmigr_cpu);
 
+static bool tmigr_enabled = true;
+
 #define TMIGR_NONE	0xFF
 #define BIT_CNT		8
 
@@ -1790,6 +1792,9 @@ static int __init tmigr_init(void)
 
 	BUILD_BUG_ON_NOT_POWER_OF_2(TMIGR_CHILDREN_PER_GROUP);
 
+	if (!tmigr_enabled)
+		return 0;
+
 	/* Nothing to do if running on UP */
 	if (ncpus == 1)
 		return 0;
@@ -1854,3 +1859,17 @@ static int __init tmigr_init(void)
 	return ret;
 }
 early_initcall(tmigr_init);
+
+static int __init tmigr_setup(char *str)
+{
+	if (!str)
+		return 0;
+
+	if (!strcmp(str, "off"))
+		tmigr_enabled = false;
+	else if (!strcmp(str, "on"))
+		tmigr_enabled = true;
+
+	return 0;
+}
+early_param("tmigr", tmigr_setup);
-- 
2.49.0

Re: [PATCH v2 1/1] timers/migration: add 'tmigr' kernel parameter to optionally disable timer migration

[Randy Dunlap]

Hi,

On 9/10/25 12:42 AM, Sehee Jeong wrote:
> On heterogeneous systems with big.LITTLE architectures, timer migration
> may cause timers from little cores to run on big cores, or vice versa,
> because core type differences are not considered in the current timer
> migration logic. This can be undesirable in systems that require
> strict power management.
> 
> For example, if timers are frequently migrated to a big CPU, it must
> handle callbacks that could have run on a little CPU. This reduces the
> big CPU's idle residency and increases overall energy consumption due to
> higher power draw on the big CPU.
> 
> To avoid this issue, introduce an early boot parameter to optionally
> disable timer migration:
> 
>     tmigr=on|off	(default: on)
> 
> When set to "off", timer migration initialization is skipped entirely.
> 
> Signed-off-by: Sehee Jeong <sehee1.jeong@samsung.com>
> ---
>  .../admin-guide/kernel-parameters.txt         |  4 ++++
>  kernel/time/timer_migration.c                 | 19 +++++++++++++++++++
>  2 files changed, 23 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index ab4c049faba9..0f5d42c046bb 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -6783,6 +6783,10 @@
>  			Force threading of all interrupt handlers except those
>  			marked explicitly IRQF_NO_THREAD.
>  
> +	tmigr		[KNL,EARLY] Enable/disable timer migration

	tmigr=
please.

> +			Valid parameters: on, off
> +			Default: on
> +
>  	topology=	[S390,EARLY]
>  			Format: {off | on}
>  			Specify if the kernel should make use of the cpu
> diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c
> index 72538baa7a1f..00e3740aec98 100644
> --- a/kernel/time/timer_migration.c
> +++ b/kernel/time/timer_migration.c
> @@ -422,6 +422,8 @@ static unsigned int tmigr_crossnode_level __read_mostly;
>  
>  static DEFINE_PER_CPU(struct tmigr_cpu, tmigr_cpu);
>  
> +static bool tmigr_enabled = true;
> +
>  #define TMIGR_NONE	0xFF
>  #define BIT_CNT		8
>  
> @@ -1790,6 +1792,9 @@ static int __init tmigr_init(void)
>  
>  	BUILD_BUG_ON_NOT_POWER_OF_2(TMIGR_CHILDREN_PER_GROUP);
>  
> +	if (!tmigr_enabled)
> +		return 0;
> +
>  	/* Nothing to do if running on UP */
>  	if (ncpus == 1)
>  		return 0;
> @@ -1854,3 +1859,17 @@ static int __init tmigr_init(void)
>  	return ret;
>  }
>  early_initcall(tmigr_init);
> +
> +static int __init tmigr_setup(char *str)
> +{
> +	if (!str)
> +		return 0;
> +
> +	if (!strcmp(str, "off"))
> +		tmigr_enabled = false;
> +	else if (!strcmp(str, "on"))
> +		tmigr_enabled = true;
> +
> +	return 0;
> +}
> +early_param("tmigr", tmigr_setup);

Re: [PATCH v2 0/1] timers/migration: add 'tmigr' kernel parameter to optionally disable timer migration

[Thomas Gleixner]

On Wed, Sep 10 2025 at 16:42, Sehee Jeong wrote:
> This patch introduces a kernel boot parameter to optionally disable the
> timer migration.
>
> On heterogeneous systems with big.LITTLE architectures, timer migration
> may cause timers from little cores to run on big cores, or vice versa,
> because core type differences are not considered in the current timer
> migration logic.
>
> In our heterogeneous system, we observed timers being migrated to big
> CPU frequently, resulting in timer callbacks that could have run on
> little CPUs being executed on the big CPU instead. This reduced big
> CPU's idle residency and increased overall power consumption due to
> higher power draw on the big CPU. Since timer_migration is a relatively
> new feature, addressing the structural limitation was difficult.

It's not that new, but anyway I'm not understanding what's so difficult
to address that problem in the migration code. As all of that code is
based on a hierarchy, so it requires to ensure that:

   All big CPUs are on one side of the hierarchy and all little CPUs
   on the other side and those sides are not connected.

Taking the example from the comment on top of the code:

 * LVL 2                           [GRP2:0]
 *                              GRP1:0 = GRP1:M
 *
 * LVL 1            [GRP1:0]                      [GRP1:1]
 *               GRP0:0 - GRP0:2               GRP0:3 - GRP0:5
 *
 * LVL 0  [GRP0:0]  [GRP0:1]  [GRP0:2]  [GRP0:3]  [GRP0:4]  [GRP0:5]
 * CPUS     0-7       8-15      16-23     24-31     32-39     40-47

Assume that GRP1:0 is the big cluster and GRP1:1 is the little
cluster. Then obviously LVL2 is not required for this scenario. So the
resulting hierarchy looks like this:

 * LVL 1            [GRP1:0]                      [GRP1:1]
 *               GRP0:0 - GRP0:2               GRP0:3 - GRP0:5
 *
 * LVL 0  [GRP0:0]  [GRP0:1]  [GRP0:2]  [GRP0:3]  [GRP0:4]  [GRP0:5]
 * CPUS     0-7       8-15      16-23     24-31     32-39     40-47

It works nicely even when the clusters are asymetric, i.e. one requires
more levels than the other:

 * LVL 1            [GRP1:0]                      [GRP1:1]
 *               GRP0:0		               GRP0:1 - GRP0:3
 *
 * LVL 0  [GRP0:0]                        [GRP0:1]  [GRP0:2]  [GRP0:3]
 * CPUS     0-7                             8-15      16-23     24-31 

GRP1:0 is just there to keep the code simple, but as the hierarchy ends
there it is not any different than having only CPU 8-15 online in
GRP1:1.

That means the required changes boil down to:

  1) Calculate the hierarchy levels based on big/little clusters and not
     connect those clusters on top in tmigr_init(), which means the
     hierarchy levels are one less than on a connected system.

  2) Ensure that the CPUs end up on the correct side of the hierarchy in
     tmigr_setup_groups()

  3) Prevent tmigr_cpu_offline() from assigning a cross cluster migrator.

Everything else works out of the box.

I obviously might be missing the real difficulty you observed when you
tried to address this structural "limitation". You surely can explain
that to me then, right?

> Therefore, this patch adds a boot parameter to optionally disable timer
> migration.

You still did not address my comment, that this also means that all
timers must expire on the CPU they are armed on. Which in turn causes
less idle recidency in the overall system, no?

Thanks

        tglx