From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5198D301024 for ; Tue, 9 Dec 2025 16:36:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765298181; cv=none; b=Yc1R/3yoXQmMQdIdq8Z7CgS5+LXgmdJlTFV0O8f2KfFNI/K/7FshI22KNvniHSe+BQsfDec5V7q3IFpNCrH4f48AUvo4bRPdhQ/rOGRu+FhWWSbe0KOmMRtoXHos2gA/HOMI9fDzSJnfBzwjf52FXQqWFEC2zFCmK1Rup35RNMM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765298181; c=relaxed/simple; bh=Fqfe5FgxVs+MG6sXahKJfbQOm1YW996lIRNSDKsAEec=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=eH0gieVhnk8bWy0Zm4MxlP/vFHY3qHcD+h1aIz4DkyH1bd2dK/8+2vIGW89zHrJaDRXgeY+fgLLWM1SEONHaQhNFwjLDp0fm6GbulPBBl63MUuj1aVSGnfP/TzvrDEtKs//XMlH8t4v/hOJofLHA6LwOJGdTfPnXwQzt8SF+vNg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 117C4175D; Tue, 9 Dec 2025 08:36:11 -0800 (PST) Received: from [10.1.196.46] (e134344.arm.com [10.1.196.46]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DD1D93F73B; Tue, 9 Dec 2025 08:36:14 -0800 (PST) Message-ID: Date: Tue, 9 Dec 2025 16:36:13 +0000 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH 31/38] arm_mpam: resctrl: Update the rmid reallocation limit To: Fenghua Yu , James Morse , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Dave Martin , Koba Ko , Shanker Donthineni , baisheng.gao@unisoc.com, Jonathan Cameron , Gavin Shan , rohit.mathew@arm.com, reinette.chatre@intel.com, Punit Agrawal References: <20251205215901.17772-1-james.morse@arm.com> <20251205215901.17772-32-james.morse@arm.com> <1cc804d5-9d14-4d71-9da1-a660a2569e4e@nvidia.com> From: Ben Horgan Content-Language: en-US In-Reply-To: <1cc804d5-9d14-4d71-9da1-a660a2569e4e@nvidia.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi Fenghua, On 12/6/25 00:06, Fenghua Yu wrote: > Hi, James, > > On 12/5/25 13:58, James Morse wrote: >> resctrl's limbo code needs to be told when the data left in a cache is >> small enough for the partid+pmg value to be re-allocated. >> >> x86 uses the cache size divided by the number of rmid users the cache >> may have. Do the same, but for the smallest cache, and with the >> number of partid-and-pmg users. >> >> Querying the cache size can't happen until after cacheinfo_sysfs_init() >> has run, so mpam_resctrl_setup() must wait until then. >> >> Signed-off-by: James Morse >> --- >>   drivers/resctrl/mpam_resctrl.c | 54 ++++++++++++++++++++++++++++++++++ >>   1 file changed, 54 insertions(+) >> >> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/ >> mpam_resctrl.c >> index 506063bd3348..ccdf8db742c9 100644 >> --- a/drivers/resctrl/mpam_resctrl.c >> +++ b/drivers/resctrl/mpam_resctrl.c >> @@ -16,6 +16,7 @@ >>   #include >>   #include >>   #include >> +#include >>     #include >>   @@ -58,6 +59,13 @@ static bool cdp_enabled; >>    */ >>   static bool resctrl_enabled; >>   +/* >> + * mpam_resctrl_pick_caches() needs to know the size of the caches. >> cacheinfo >> + * populates this from a device_initcall(). mpam_resctrl_setup() must >> wait. >> + */ >> +static bool cacheinfo_ready; >> +static DECLARE_WAIT_QUEUE_HEAD(wait_cacheinfo_ready); >> + >>   /* >>    * L3 local/total may come from different classes - what is the >> number of MBWU >>    * 'on L3'? >> @@ -584,6 +592,38 @@ void resctrl_arch_reset_cntr(struct rdt_resource >> *r, struct rdt_mon_domain *d, >>       reset_mon_cdp_safe(mon, mon_comp, USE_PRE_ALLOCATED, closid, rmid); >>   } >>   +/* >> + * The rmid realloc threshold should be for the smallest cache >> exposed to >> + * resctrl. >> + */ >> +static int update_rmid_limits(struct mpam_class *class) >> +{ >> +    u32 num_unique_pmg = resctrl_arch_system_num_rmid_idx(); >> +    struct mpam_props *cprops = &class->props; >> +    struct cacheinfo *ci; >> + >> +    lockdep_assert_cpus_held(); >> + >> +    /* Assume cache levels are the same size for all CPUs... */ >> +    ci = get_cpu_cacheinfo_level(smp_processor_id(), class->level); >> +    if (!ci || ci->size == 0) { >> +        pr_debug("Could not read cache size for class %u\n", >> +             class->level); >> +        return -EINVAL; >> +    } >> + >> +    if (!mpam_has_feature(mpam_feat_msmon_csu, cprops)) >> +        return 0; >> + >> +    if (!resctrl_rmid_realloc_limit || >> +        ci->size < resctrl_rmid_realloc_limit) { >> +        resctrl_rmid_realloc_limit = ci->size; >> +        resctrl_rmid_realloc_threshold = ci->size / num_unique_pmg; >> +    } >> + >> +    return 0; >> +} >> + >>   static bool cache_has_usable_cpor(struct mpam_class *class) >>   { >>       struct mpam_props *cprops = &class->props; >> @@ -1025,6 +1065,9 @@ static void mpam_resctrl_pick_counters(void) >>               /* CSU counters only make sense on a cache. */ >>               switch (class->type) { >>               case MPAM_CLASS_CACHE: >> +                if (update_rmid_limits(class)) >> +                    continue; >> + >>                   counter_update_class(QOS_L3_OCCUP_EVENT_ID, class); >>                   return; >>               default: >> @@ -1731,6 +1774,8 @@ int mpam_resctrl_setup(void) >>       struct mpam_resctrl_res *res; >>       struct mpam_resctrl_mon *mon; >>   +    wait_event(wait_cacheinfo_ready, cacheinfo_ready); > > This may cause system hang for any hw/fw issue that causes cacheinfo > failure. Instead of hang, is it better to have a timeout wait here? Like >     errowait_event_timeout(wait_cache_info_read, cacheinfo_ready, 5 * HZ); > and report failure when cacheinfo is not ready. This is just waiting on everything in device_initcall(). I think we've got bigger problems if that doesn't finish. > > Thanks. > > -Fenghua Thanks, Ben