Re: [PATCH 3/3] arm64: dts: qcom: qcm2290: Hook up MPM

[Marek Szyprowski <m.szyprowski@samsung.com>]

On 15.12.2023 01:01, Konrad Dybcio wrote:
> Wire up MPM and the interrupts it provides.
>
> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
> ---
>  arch/arm64/boot/dts/qcom/qcm2290.dtsi | 42 +++++++++++++++++++++++++++++------
>  1 file changed, 35 insertions(+), 7 deletions(-)
>
> diff --git a/arch/arm64/boot/dts/qcom/qcm2290.dtsi b/arch/arm64/boot/dts/qcom/qcm2290.dtsi
> index ce04d0acdede..0911fb08ed63 100644
> --- a/arch/arm64/boot/dts/qcom/qcm2290.dtsi
> +++ b/arch/arm64/boot/dts/qcom/qcm2290.dtsi
> @@ -199,6 +199,7 @@ CPU_PD3: power-domain-cpu3 {
>  
>  		CLUSTER_PD: power-domain-cpu-cluster {
>  			#power-domain-cells = <0>;
> +			power-domains = <&mpm>;
>  			domain-idle-states = <&CLUSTER_SLEEP>;
>  		};
>  	};


This patch landed long time ago in mainline as commit e3f6a6994041
("arm64: dts: qcom: qcm2290: Hook up MPM").

Recently I've added Arduino UnoQ board
(arch/arm64/boot/dts/qcom/qrb2210-arduino-imola.dts) to my test farm
and found that this patch is responsible for the "Invalid wait context"
warning [1] observed during boot of the recent kernels when
CONFIG_PROVE_RAW_LOCK_NESTING is set (which is implicitly true when
CONFIG_PROVE_LOCKING is set since commit d8fccd9ca5f90). This shows
a real problem if one wants to run RT-enabled kernel.

This is because the above chunk adds the "mpm" as a parent power domain
to the "cluster_pd" (in current arch/arm64/boot/dts/qcom/agatti.dtsi). 
"cluster_pd" is initialized as "GENPD_FLAG_IRQ_SAFE | GENPD_FLAG_CPU_DOMAIN"
(see psci_pd_init in drivers/cpuidle/cpuidle-psci-domain.c, which use
raw_spinlock_t based locking), while the "mpm" power domain only as 
"GENPD_FLAG_IRQ_SAFE" (see qcom_mpm_probe in drivers/irqchip/irq-qcom-mpm.c,
which use spinlock_t locking).

I've check and there is no easy way to make the "mpm" power domain
GENPD_FLAG_CPU_DOMAIN compatible. drivers/irqchip/irq-qcom-mpm.c driver
uses mbox API, which internally uses spinlock_t based locking. Then
there is a drivers/mailbox/qcom-apcs-ipc-mailbox.c driver and even
more dependencies: mmio regmap, clocks and probably more (I've didn't
check further).

The question is how to solve this issue? Is this dependency really
required? Would it work if the "mpm" power domain is set as always
on and genpd framework extended to support such case without
triggering lockdep warning (it already has support for irq-safe
device in non-irq-safe power domain)?



[1] mentioned kernel log :

 =============================
 [ BUG: Invalid wait context ]
 7.2.0-rc2-next-20260708+ #13154 Not tainted
 -----------------------------
 kworker/u16:1/44 is trying to lock:
 ffff000002517870 (&genpd->slock/1){....}-{3:3}, at: genpd_lock_nested_spin+0x1c/0x30
 other info that might help us debug this:
 context-{5:5}
 locks held by kworker/u16:1/44: 5, last CPU#3:
  #0: ffff000000032140 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x6a8/0x86c
  #1: ffff800080533d90 (deferred_probe_work){+.+.}-{0:0}, at: process_one_work+0x1f8/0x86c
  #2: ffff00000036c138 (&dev->mutex){....}-{4:4}, at: __device_attach+0x3c/0x1b8
  #3: ffffa850a0fca5f8 (gpd_list_lock){+.+.}-{4:4}, at: of_genpd_add_subdomain+0x30/0xc8
  #4: ffff000001994750 (&genpd->raw_slock){....}-{2:2}, at: genpd_lock_raw_spin+0x1c/0x30
 stack backtrace:
 CPU: 3 UID: 0 PID: 44 Comm: kworker/u16:1 Not tainted 7.2.0-rc2-next-20260708+ #13154 PREEMPT
 Hardware name: Arduino UnoQ (DT)
 Workqueue: events_unbound deferred_probe_work_func
 Call trace:
  show_stack+0x18/0x24 (C)
  dump_stack_lvl+0x90/0xd0
  dump_stack+0x18/0x24
  __lock_acquire+0xa78/0x22a8
  lock_acquire+0x1d8/0x404
  _raw_spin_lock_irqsave_nested+0x64/0x8c
  genpd_lock_nested_spin+0x1c/0x30
  genpd_add_subdomain+0x8c/0x1d8
  of_genpd_add_subdomain+0x68/0xc8
  dt_idle_pd_init_topology+0x88/0x100
  psci_cpuidle_domain_probe+0x1bc/0x31c
  platform_probe+0x5c/0xac
  really_probe+0xbc/0x298
  __driver_probe_device+0x164/0x184
  driver_probe_device+0xbc/0x11c
  __device_attach_driver+0xb8/0x118
  bus_for_each_drv+0x84/0xe4
  __device_attach+0xac/0x1b8
  device_initial_probe+0x50/0x54
  bus_probe_device+0x38/0xa8
  deferred_probe_work_func+0x8c/0xc8
  process_one_work+0x248/0x86c
  worker_thread+0x240/0x388
  kthread+0x13c/0x148
  ret_from_fork+0x10/0x20
 CPUidle PSCI: Initialized CPU PM domain topology using OSI mode



> @@ -266,6 +267,24 @@ rpmpd_opp_turbo_plus: opp8 {
>  				};
>  			};
>  		};
> +
> +		mpm: interrupt-controller {
> +			compatible = "qcom,mpm";
> +			qcom,rpm-msg-ram = <&apss_mpm>;
> +			interrupts = <GIC_SPI 197 IRQ_TYPE_EDGE_RISING>;
> +			mboxes = <&apcs_glb 1>;
> +			interrupt-controller;
> +			#interrupt-cells = <2>;
> +			#power-domain-cells = <0>;
> +			interrupt-parent = <&intc>;
> +			qcom,mpm-pin-count = <96>;
> +			qcom,mpm-pin-map = <2 275>,  /* TSENS0 uplow */
> +					   <5 296>,  /* Soundwire master_irq */
> +					   <12 422>, /* DWC3 ss_phy_irq */
> +					   <24 79>,  /* Soundwire wake_irq */
> +					   <86 183>, /* MPM wake, SPMI */
> +					   <90 260>; /* QUSB2_PHY DP+DM */
> +		};
>  	};
>  
>  	reserved_memory: reserved-memory {
> @@ -429,6 +448,7 @@ tlmm: pinctrl@500000 {
>  			interrupts = <GIC_SPI 227 IRQ_TYPE_LEVEL_HIGH>;
>  			gpio-controller;
>  			gpio-ranges = <&tlmm 0 0 127>;
> +			wakeup-parent = <&mpm>;
>  			#gpio-cells = <2>;
>  			interrupt-controller;
>  			#interrupt-cells = <2>;
> @@ -778,7 +798,7 @@ spmi_bus: spmi@1c40000 {
>  				    "obsrvr",
>  				    "intr",
>  				    "cnfg";
> -			interrupts = <GIC_SPI 183 IRQ_TYPE_LEVEL_HIGH>;
> +			interrupts-extended = <&mpm 86 IRQ_TYPE_LEVEL_HIGH>;
>  			interrupt-names = "periph_irq";
>  			qcom,ee = <0>;
>  			qcom,channel = <0>;
> @@ -793,8 +813,8 @@ tsens0: thermal-sensor@4411000 {
>  			reg = <0x0 0x04411000 0x0 0x1ff>,
>  			      <0x0 0x04410000 0x0 0x8>;
>  			#qcom,sensors = <10>;
> -			interrupts = <GIC_SPI 275 IRQ_TYPE_LEVEL_HIGH>,
> -				     <GIC_SPI 190 IRQ_TYPE_LEVEL_HIGH>;
> +			interrupts-extended = <&mpm 2 IRQ_TYPE_LEVEL_HIGH>,
> +					      <&intc GIC_SPI 190 IRQ_TYPE_LEVEL_HIGH>;
>  			interrupt-names = "uplow", "critical";
>  			#thermal-sensor-cells = <1>;
>  		};
> @@ -813,8 +833,15 @@ bimc: interconnect@4480000 {
>  		};
>  
>  		rpm_msg_ram: sram@45f0000 {
> -			compatible = "qcom,rpm-msg-ram";
> +			compatible = "qcom,rpm-msg-ram", "mmio-sram";
>  			reg = <0x0 0x045f0000 0x0 0x7000>;
> +			#address-cells = <1>;
> +			#size-cells = <1>;
> +			ranges = <0 0x0 0x045f0000 0x7000>;
> +
> +			apss_mpm: sram@1b8 {
> +				reg = <0x1b8 0x48>;
> +			};
>  		};
>  
>  		sram@4690000 {
> @@ -1293,9 +1320,10 @@ &qup_virt SLAVE_QUP_CORE_0 RPM_ALWAYS_TAG>,
>  		usb: usb@4ef8800 {
>  			compatible = "qcom,qcm2290-dwc3", "qcom,dwc3";
>  			reg = <0x0 0x04ef8800 0x0 0x400>;
> -			interrupts = <GIC_SPI 260 IRQ_TYPE_LEVEL_HIGH>,
> -				     <GIC_SPI 422 IRQ_TYPE_LEVEL_HIGH>;
> -			interrupt-names = "hs_phy_irq", "ss_phy_irq";
> +			interrupts-extended = <&intc GIC_SPI 260 IRQ_TYPE_LEVEL_HIGH>,
> +					      <&mpm 12 IRQ_TYPE_LEVEL_HIGH>;
> +			interrupt-names = "hs_phy_irq",
> +					  "ss_phy_irq";
>  
>  			clocks = <&gcc GCC_CFG_NOC_USB3_PRIM_AXI_CLK>,
>  				 <&gcc GCC_USB30_PRIM_MASTER_CLK>,
>
Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland