[PATCH v3 0/2] Fixes for UFS clock gating initialization

[PATCH v3 0/2] Fixes for UFS clock gating initialization

[Avri Altman]

Martin hi,

This patch series addresses two issues related to the UFS clock gating
mechanism. The first patch ensures that the `clk_gating.lock` is used
only after it has been properly initialized.

The second patch fixes an issue where `clk_gating.state` is toggled even
if clock gating is not allowed, which can lead to crashes.

Changes since v2:
 - Add patch #2 (Geert)
 - Initialize clk_gating.lock unconditionally (Bart)

Changes since v1:
 - move the spin_lock_init(&hba->clk_gating.lock) call from
   ufshcd_init_clk_gating() just before the ufshcd_hba_init() call in
   ufshcd_init() (Bart)

Avri Altman (2):
  scsi: ufs: core: Ensure clk_gating.lock is used only after
    initialization
  scsi: ufs: Fix toggling of clk_gating.state when clock gating is not
    allowed

 drivers/ufs/core/ufshcd.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

-- 
2.25.1

[PATCH v3 1/2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Avri Altman]

This commit addresses a lockdep warning triggered by the use of the
clk_gating.lock before it is properly initialized. The warning is as
follows:

[    4.388838] INFO: trying to register non-static key.
[    4.395673] The code is fine but needs lockdep annotation, or maybe
[    4.402118] you didn't initialize this object before use?
[    4.407673] turning off the locking correctness validator.
[    4.413334] CPU: 5 UID: 0 PID: 58 Comm: kworker/u32:1 Not tainted 6.12-rc1 #185
[    4.413343] Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT)
[    4.413362] Call trace:
[    4.413364]  show_stack+0x18/0x24 (C)
[    4.413374]  dump_stack_lvl+0x90/0xd0
[    4.413384]  dump_stack+0x18/0x24
[    4.413392]  register_lock_class+0x498/0x4a8
[    4.413400]  __lock_acquire+0xb4/0x1b90
[    4.413406]  lock_acquire+0x114/0x310
[    4.413413]  _raw_spin_lock_irqsave+0x60/0x88
[    4.413423]  ufshcd_setup_clocks+0x2c0/0x490
[    4.413433]  ufshcd_init+0x198/0x10ec
[    4.413437]  ufshcd_pltfrm_init+0x600/0x7c0
[    4.413444]  ufs_qcom_probe+0x20/0x58
[    4.413449]  platform_probe+0x68/0xd8
[    4.413459]  really_probe+0xbc/0x268
[    4.413466]  __driver_probe_device+0x78/0x12c
[    4.413473]  driver_probe_device+0x40/0x11c
[    4.413481]  __device_attach_driver+0xb8/0xf8
[    4.413489]  bus_for_each_drv+0x84/0xe4
[    4.413495]  __device_attach+0xfc/0x18c
[    4.413502]  device_initial_probe+0x14/0x20
[    4.413510]  bus_probe_device+0xb0/0xb4
[    4.413517]  deferred_probe_work_func+0x8c/0xc8
[    4.413524]  process_scheduled_works+0x250/0x658
[    4.413534]  worker_thread+0x15c/0x2c8
[    4.413542]  kthread+0x134/0x200
[    4.413550]  ret_from_fork+0x10/0x20

To fix this issue, ensure that the spinlock is only used after it
has been properly initialized before using it in `ufshcd_setup_clocks`.
Do that unconditionally as initializing a spinlock is a fast operation.

Fixes: 209f4e43b806 ("scsi: ufs: core: Introduce a new clock_gating lock")
Reported-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Avri Altman <avri.altman@wdc.com>
---
 drivers/ufs/core/ufshcd.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index f6c38cf10382..b73c87da383d 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -2120,8 +2120,6 @@ static void ufshcd_init_clk_gating(struct ufs_hba *hba)
 	INIT_DELAYED_WORK(&hba->clk_gating.gate_work, ufshcd_gate_work);
 	INIT_WORK(&hba->clk_gating.ungate_work, ufshcd_ungate_work);
 
-	spin_lock_init(&hba->clk_gating.lock);
-
 	hba->clk_gating.clk_gating_workq = alloc_ordered_workqueue(
 		"ufs_clk_gating_%d", WQ_MEM_RECLAIM | WQ_HIGHPRI,
 		hba->host->host_no);
@@ -10412,6 +10410,12 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq)
 	hba->irq = irq;
 	hba->vps = &ufs_hba_vps;
 
+	/*
+	 * Initialize clk_gating.lock early since it is being used in
+	 * ufshcd_setup_clocks()
+	 */
+	spin_lock_init(&hba->clk_gating.lock);
+
 	err = ufshcd_hba_init(hba);
 	if (err)
 		goto out_error;
-- 
2.25.1

Re: [PATCH v3 1/2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Bean Huo]

Reviewed-by: Bean Huo <beanhuo@micron.com>

Re: [PATCH v3 1/2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Geert Uytterhoeven]

Hi Avri,

On Sun, 26 Jan 2025 at 07:48, Avri Altman <avri.altman@wdc.com> wrote:
> This commit addresses a lockdep warning triggered by the use of the
> clk_gating.lock before it is properly initialized. The warning is as
> follows:
>
> [    4.388838] INFO: trying to register non-static key.
> [    4.395673] The code is fine but needs lockdep annotation, or maybe
> [    4.402118] you didn't initialize this object before use?
> [    4.407673] turning off the locking correctness validator.
> [    4.413334] CPU: 5 UID: 0 PID: 58 Comm: kworker/u32:1 Not tainted 6.12-rc1 #185
> [    4.413343] Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT)
> [    4.413362] Call trace:
> [    4.413364]  show_stack+0x18/0x24 (C)
> [    4.413374]  dump_stack_lvl+0x90/0xd0
> [    4.413384]  dump_stack+0x18/0x24
> [    4.413392]  register_lock_class+0x498/0x4a8
> [    4.413400]  __lock_acquire+0xb4/0x1b90
> [    4.413406]  lock_acquire+0x114/0x310
> [    4.413413]  _raw_spin_lock_irqsave+0x60/0x88
> [    4.413423]  ufshcd_setup_clocks+0x2c0/0x490
> [    4.413433]  ufshcd_init+0x198/0x10ec
> [    4.413437]  ufshcd_pltfrm_init+0x600/0x7c0
> [    4.413444]  ufs_qcom_probe+0x20/0x58
> [    4.413449]  platform_probe+0x68/0xd8
> [    4.413459]  really_probe+0xbc/0x268
> [    4.413466]  __driver_probe_device+0x78/0x12c
> [    4.413473]  driver_probe_device+0x40/0x11c
> [    4.413481]  __device_attach_driver+0xb8/0xf8
> [    4.413489]  bus_for_each_drv+0x84/0xe4
> [    4.413495]  __device_attach+0xfc/0x18c
> [    4.413502]  device_initial_probe+0x14/0x20
> [    4.413510]  bus_probe_device+0xb0/0xb4
> [    4.413517]  deferred_probe_work_func+0x8c/0xc8
> [    4.413524]  process_scheduled_works+0x250/0x658
> [    4.413534]  worker_thread+0x15c/0x2c8
> [    4.413542]  kthread+0x134/0x200
> [    4.413550]  ret_from_fork+0x10/0x20
>
> To fix this issue, ensure that the spinlock is only used after it
> has been properly initialized before using it in `ufshcd_setup_clocks`.
> Do that unconditionally as initializing a spinlock is a fast operation.
>
> Fixes: 209f4e43b806 ("scsi: ufs: core: Introduce a new clock_gating lock")
> Reported-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
> Signed-off-by: Avri Altman <avri.altman@wdc.com>

Thank you, this fixes the similar BUG (s/ufs_qcom_probe/ufs_renesas_probe/)
I was seeing on the Renesas R-Car S4 Starter Kit.
Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

[PATCH v3 2/2] scsi: ufs: Fix toggling of clk_gating.state when clock gating is not allowed

[Avri Altman]

This commit addresses an issue where `clk_gating.state` is being toggled
in `ufshcd_setup_clocks` even if clock gating is not allowed. This can
lead to a crash with the following error:

    BUG: spinlock bad magic on CPU#6, swapper/0/1
     lock: 0xffffff84443014e8, .magic: 00000000, .owner: <none>/-1,
    .owner_cpu: 0
    CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Not tainted
    6.13.0-rcar3-initrd-08318-g75abbef32a94 #896
    Hardware name: R-Car S4 Starter Kit board (DT)
    Call trace:
     show_stack+0x18/0x24 (C)
     dump_stack_lvl+0x60/0x80
     dump_stack+0x18/0x24
     spin_bug+0x7c/0xa0
     do_raw_spin_lock+0x34/0xb4
     _raw_spin_lock_irqsave+0x1c/0x30
     class_spinlock_irqsave_constructor+0x18/0x30
     ufshcd_setup_clocks+0x98/0x23c
     ufshcd_init+0x288/0xd38
     ufshcd_pltfrm_init+0x618/0x738
     ufs_renesas_probe+0x18/0x24
     platform_probe+0x68/0xb8
     really_probe+0x138/0x268
     __driver_probe_device+0xf4/0x10c
     driver_probe_device+0x3c/0xf8
     __driver_attach+0xf0/0x100
     bus_for_each_dev+0x84/0xdc
     driver_attach+0x24/0x30
     bus_add_driver+0xe8/0x1dc
     driver_register+0xbc/0xf8
     __platform_driver_register+0x24/0x30
     ufs_renesas_platform_init+0x1c/0x28
     do_one_initcall+0x84/0x1f4
     kernel_init_freeable+0x238/0x23c
     kernel_init+0x20/0x120
     ret_from_fork+0x10/0x20

The root cause of the issue is that `clk_gating.state` is being toggled
even if clock gating is not allowed. This can lead to the spinlock being
used before it is properly initialized.

The fix is to add a check for `hba->clk_gating.is_initialized` before
toggling `clk_gating.state` in `ufshcd_setup_clocks`. Since
`clk_gating.lock` is now initialized unconditionally, this is for
documentation purposes, to ensure clarity in the code. The primary fix
remains to prevent toggling the `clk_gating.state` if clock gating is
not allowed.

Fixes: 1ab27c9cf8b6 ("ufs: Add support for clock gating")
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Avri Altman <avri.altman@wdc.com>
---
 drivers/ufs/core/ufshcd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index b73c87da383d..abe0774133f5 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -9140,7 +9140,7 @@ static int ufshcd_setup_clocks(struct ufs_hba *hba, bool on)
 			if (!IS_ERR_OR_NULL(clki->clk) && clki->enabled)
 				clk_disable_unprepare(clki->clk);
 		}
-	} else if (!ret && on) {
+	} else if (!ret && on && hba->clk_gating.is_initialized) {
 		scoped_guard(spinlock_irqsave, &hba->clk_gating.lock)
 			hba->clk_gating.state = CLKS_ON;
 		trace_ufshcd_clk_gating(dev_name(hba->dev),
-- 
2.25.1

Re: [PATCH v3 2/2] scsi: ufs: Fix toggling of clk_gating.state when clock gating is not allowed

[Bean Huo]

On Sun, 2025-01-26 at 08:45 +0200, Avri Altman wrote:
>         } else if (!ret && on && hba->clk_gating.is_initialized) {
>                 scoped_guard(spinlock_irqsave, &hba->clk_gating.lock)
>                         hba->clk_gating.state = CLKS_ON;
>                 trace_ufshcd_clk_gating(dev_name(hba->dev)

Consolidating all clock-related initialization into a single function,
such as ufshcd_init_clocks(), should be considered.

RE: [PATCH v3 2/2] scsi: ufs: Fix toggling of clk_gating.state when clock gating is not allowed

[Avri Altman]

> On Sun, 2025-01-26 at 08:45 +0200, Avri Altman wrote:
> >         } else if (!ret && on && hba->clk_gating.is_initialized) {
> >                 scoped_guard(spinlock_irqsave, &hba->clk_gating.lock)
> >                         hba->clk_gating.state = CLKS_ON;
> >                 trace_ufshcd_clk_gating(dev_name(hba->dev)
> 
> Consolidating all clock-related initialization into a single function, such as
> ufshcd_init_clocks(), should be considered.
Hi Bean,
I don't think it will resolve the issue that is addressed in this patch.

However, consolidating all clock-related initialization into a single function, is indeed a good idea.
I might try to do that - but in another patch.

Thanks,
Avri

Re: [PATCH v3 2/2] scsi: ufs: Fix toggling of clk_gating.state when clock gating is not allowed

[Geert Uytterhoeven]

Hi Avri,

Thanks for your patch!

On Sun, 26 Jan 2025 at 07:48, Avri Altman <avri.altman@wdc.com> wrote:
> This commit addresses an issue where `clk_gating.state` is being toggled
> in `ufshcd_setup_clocks` even if clock gating is not allowed. This can
> lead to a crash with the following error:
>
>     BUG: spinlock bad magic on CPU#6, swapper/0/1
>      lock: 0xffffff84443014e8, .magic: 00000000, .owner: <none>/-1,
>     .owner_cpu: 0
>     CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Not tainted
>     6.13.0-rcar3-initrd-08318-g75abbef32a94 #896
>     Hardware name: R-Car S4 Starter Kit board (DT)
>     Call trace:
>      show_stack+0x18/0x24 (C)
>      dump_stack_lvl+0x60/0x80
>      dump_stack+0x18/0x24
>      spin_bug+0x7c/0xa0
>      do_raw_spin_lock+0x34/0xb4
>      _raw_spin_lock_irqsave+0x1c/0x30
>      class_spinlock_irqsave_constructor+0x18/0x30
>      ufshcd_setup_clocks+0x98/0x23c
>      ufshcd_init+0x288/0xd38
>      ufshcd_pltfrm_init+0x618/0x738
>      ufs_renesas_probe+0x18/0x24
>      platform_probe+0x68/0xb8
>      really_probe+0x138/0x268
>      __driver_probe_device+0xf4/0x10c
>      driver_probe_device+0x3c/0xf8
>      __driver_attach+0xf0/0x100
>      bus_for_each_dev+0x84/0xdc
>      driver_attach+0x24/0x30
>      bus_add_driver+0xe8/0x1dc
>      driver_register+0xbc/0xf8
>      __platform_driver_register+0x24/0x30
>      ufs_renesas_platform_init+0x1c/0x28
>      do_one_initcall+0x84/0x1f4
>      kernel_init_freeable+0x238/0x23c
>      kernel_init+0x20/0x120
>      ret_from_fork+0x10/0x20

Note that after "[PATCH v3 1/2] scsi: ufs: core: Ensure clk_gating.lock
is used only after initialization", I no longer see the above BUG(), so
I don't think it's a good idea to include this log.

> The root cause of the issue is that `clk_gating.state` is being toggled
> even if clock gating is not allowed. This can lead to the spinlock being
> used before it is properly initialized.

Which doesn't mean the underlying issue no longer exists...

> The fix is to add a check for `hba->clk_gating.is_initialized` before
> toggling `clk_gating.state` in `ufshcd_setup_clocks`. Since
> `clk_gating.lock` is now initialized unconditionally, this is for
> documentation purposes, to ensure clarity in the code. The primary fix
> remains to prevent toggling the `clk_gating.state` if clock gating is
> not allowed.
>
> Fixes: 1ab27c9cf8b6 ("ufs: Add support for clock gating")
> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
> Signed-off-by: Avri Altman <avri.altman@wdc.com>

Seems to work fine on R-Car S4 SK, so
Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

RE: [PATCH v3 2/2] scsi: ufs: Fix toggling of clk_gating.state when clock gating is not allowed

[Avri Altman]

> Hi Avri,
> 
> Thanks for your patch!
> 
> On Sun, 26 Jan 2025 at 07:48, Avri Altman <avri.altman@wdc.com> wrote:
> > This commit addresses an issue where `clk_gating.state` is being
> > toggled in `ufshcd_setup_clocks` even if clock gating is not allowed.
> > This can lead to a crash with the following error:
> >
> >     BUG: spinlock bad magic on CPU#6, swapper/0/1
> >      lock: 0xffffff84443014e8, .magic: 00000000, .owner: <none>/-1,
> >     .owner_cpu: 0
> >     CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> >     6.13.0-rcar3-initrd-08318-g75abbef32a94 #896
> >     Hardware name: R-Car S4 Starter Kit board (DT)
> >     Call trace:
> >      show_stack+0x18/0x24 (C)
> >      dump_stack_lvl+0x60/0x80
> >      dump_stack+0x18/0x24
> >      spin_bug+0x7c/0xa0
> >      do_raw_spin_lock+0x34/0xb4
> >      _raw_spin_lock_irqsave+0x1c/0x30
> >      class_spinlock_irqsave_constructor+0x18/0x30
> >      ufshcd_setup_clocks+0x98/0x23c
> >      ufshcd_init+0x288/0xd38
> >      ufshcd_pltfrm_init+0x618/0x738
> >      ufs_renesas_probe+0x18/0x24
> >      platform_probe+0x68/0xb8
> >      really_probe+0x138/0x268
> >      __driver_probe_device+0xf4/0x10c
> >      driver_probe_device+0x3c/0xf8
> >      __driver_attach+0xf0/0x100
> >      bus_for_each_dev+0x84/0xdc
> >      driver_attach+0x24/0x30
> >      bus_add_driver+0xe8/0x1dc
> >      driver_register+0xbc/0xf8
> >      __platform_driver_register+0x24/0x30
> >      ufs_renesas_platform_init+0x1c/0x28
> >      do_one_initcall+0x84/0x1f4
> >      kernel_init_freeable+0x238/0x23c
> >      kernel_init+0x20/0x120
> >      ret_from_fork+0x10/0x20
> 
> Note that after "[PATCH v3 1/2] scsi: ufs: core: Ensure clk_gating.lock is used
> only after initialization", I no longer see the above BUG(), so I don't think it's
> a good idea to include this log.
Done. Will remove.

> 
> > The root cause of the issue is that `clk_gating.state` is being
> > toggled even if clock gating is not allowed. This can lead to the
> > spinlock being used before it is properly initialized.
> 
> Which doesn't mean the underlying issue no longer exists...
Done. Will rephrase.

Thanks,
Avri
> 
> > The fix is to add a check for `hba->clk_gating.is_initialized` before
> > toggling `clk_gating.state` in `ufshcd_setup_clocks`. Since
> > `clk_gating.lock` is now initialized unconditionally, this is for
> > documentation purposes, to ensure clarity in the code. The primary fix
> > remains to prevent toggling the `clk_gating.state` if clock gating is
> > not allowed.
> >
> > Fixes: 1ab27c9cf8b6 ("ufs: Add support for clock gating")
> > Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
> > Signed-off-by: Avri Altman <avri.altman@wdc.com>
> 
> Seems to work fine on R-Car S4 SK, so
> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
> 
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-
> m68k.org
> 
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like
> that.
>                                 -- Linus Torvalds