Re: [PATCH 16/33] arm_mpam: Add helpers for managing the locking around the mon_sel registers

[Dave Martin <Dave.Martin@arm.com>]

Hi,

On Fri, Aug 22, 2025 at 03:29:57PM +0000, James Morse wrote:
> The MSC MON_SEL register needs to be accessed from hardirq context by the
> PMU drivers, making an irqsave spinlock the obvious lock to protect these

What PMU drivers?  MPAM itself doesn't define its monitors as PMUs, and
(as of this series) there is no intergration with perf.

> registers. On systems with SCMI mailboxes it must be able to sleep, meaning
> a mutex must be used.
> 
> Clearly these two can't exist at the same time.

The locks obvisouly do exist at the same time.  Do you mean that an
individual MSC must be either MMIO or SCMI/PCC?

(I don't think anything prevents both kinds of MSC from existing in the
same system?)

Above, you seem to imply that each kind of MSC interface requires a
different kind of lock, but below, you imply that the locks must be
used together, with holding the outer lock being a precondition for
taking the inner lock. 

Because these functions are introduced with no user, the code doesn't
offer much in the way of clues.  In particular, there is no indication
of what the outer lock is supposed to protect.

> Add helpers for the MON_SEL locking. The outer lock must be taken in a
> pre-emptible context before the inner lock can be taken. On systems with
> SCMI mailboxes where the MON_SEL accesses must sleep - the inner lock
> will fail to be 'taken' if the caller is unable to sleep. This will allow
> the PMU driver to fail without having to check the interface type of

Why is it acceptable to fail (i.e., don't the counts need to be read on
non-MMIO MSCs?)

> each MSC.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  drivers/resctrl/mpam_internal.h | 57 ++++++++++++++++++++++++++++++++-
>  1 file changed, 56 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_internal.h
> index a623f405ddd8..c6f087f9fa7d 100644
> --- a/drivers/resctrl/mpam_internal.h
> +++ b/drivers/resctrl/mpam_internal.h
> @@ -68,10 +68,19 @@ struct mpam_msc {
>  
>  	/*
>  	 * mon_sel_lock protects access to the MSC hardware registers that are
> -	 * affeted by MPAMCFG_MON_SEL.
> +	 * affected by MPAMCFG_MON_SEL, and the mbwu_state.
> +	 * Both the 'inner' and 'outer' must be taken.
> +	 * For real MMIO MSC, the outer lock is unnecessary - but keeps the
> +	 * code common with:
> +	 * Firmware backed MSC need to sleep when accessing the MSC, which
> +	 * means some code-paths will always fail. For these MSC the outer
> +	 * lock is providing the protection, and the inner lock fails to
> +	 * be taken if the task is unable to sleep.
> +	 *
>  	 * If needed, take msc->probe_lock first.
>  	 */
>  	struct mutex		outer_mon_sel_lock;
> +	bool			outer_lock_held;

Why not use mutex_is_locked()?

>  	raw_spinlock_t		inner_mon_sel_lock;

Why raw?  The commit message makes no mention of it.

(We really to need to sit on a specific CPU while holding this lock, so
"raw" makes sense.  But we're always doing this in a cross-call,
presumably with the hotplug lock held -- so I think we can't be
migrated anyway?)

>  	unsigned long		inner_mon_sel_flags;
>  
> @@ -81,6 +90,52 @@ struct mpam_msc {
>  	struct mpam_garbage	garbage;
>  };
>  
> +static inline bool __must_check mpam_mon_sel_inner_lock(struct mpam_msc *msc)
> +{
> +	/*
> +	 * The outer lock may be taken by a CPU that then issues an IPI to run
> +	 * a helper that takes the inner lock. lockdep can't help us here.
> +	 */
> +	WARN_ON_ONCE(!msc->outer_lock_held);
> +
> +	if (msc->iface == MPAM_IFACE_MMIO) {
> +		raw_spin_lock_irqsave(&msc->inner_mon_sel_lock, msc->inner_mon_sel_flags);
> +		return true;
> +	}
> +
> +	/* Accesses must fail if we are not pre-emptible */
> +	return !!preemptible();

What accesses?

In the MPAM_IFACE_MMIO case, this returns true even though non-
preemptible (because of getting the lock).

So, what is the semantics of the return value?

A comment would probably help.

> +}
> +
> +static inline void mpam_mon_sel_inner_unlock(struct mpam_msc *msc)
> +{
> +	WARN_ON_ONCE(!msc->outer_lock_held);
> +
> +	if (msc->iface == MPAM_IFACE_MMIO)
> +		raw_spin_unlock_irqrestore(&msc->inner_mon_sel_lock, msc->inner_mon_sel_flags);
> +}
> +
> +static inline void mpam_mon_sel_outer_lock(struct mpam_msc *msc)
> +{
> +	mutex_lock(&msc->outer_mon_sel_lock);
> +	msc->outer_lock_held = true;
> +}
> +

> +static inline void mpam_mon_sel_outer_unlock(struct mpam_msc *msc)
> +{
> +	msc->outer_lock_held = false;
> +	mutex_unlock(&msc->outer_mon_sel_lock);
> +}
> +
> +static inline void mpam_mon_sel_lock_held(struct mpam_msc *msc)
> +{
> +	WARN_ON_ONCE(!msc->outer_lock_held);
> +	if (msc->iface == MPAM_IFACE_MMIO)
> +		lockdep_assert_held_once(&msc->inner_mon_sel_lock);
> +	else
> +		lockdep_assert_preemption_enabled();
> +}
> +

Except that monitors may need to be accessed in interrupt context,
I don't see an obvious difference between controls and monitors that
motivates this locking model.

Is the outer lock ever needfully held for extended periods of time,
making a (raw) spinlock unsuitable?

Cheers
---Dave