From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6154B3C13E8 for ; Fri, 20 Mar 2026 14:29:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774016946; cv=none; b=P1U7ij7QWvXuWKw8Yzgbvy+O2Ze1QI3Wmo4/9cAyRrFAYNUPSpG1ID8xVpkfj/ckXcM4iyfYMyZJ1+HeHRGVHlBfyIoOAszk+QmtpixGcoxQkyLYJM360et9mMY4iUAoI0mBj/ny1GH26jaEgfaF7qiVGHh78oxRoPNhOXU9SqE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774016946; c=relaxed/simple; bh=KeDNWQ81qSlbJc7P/NsTMgv8qsTwFMwZALvQOTPP2Cg=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=JUdC5phTpxH45kVb+GN84y+zIB8uv+L49/py6ySvHw93rHyt34OPZQfP1Y0YfXLhujBUkR+fuqxJ44u61yXGMWUVHOB6FHZ5bOOZZF0FMap9i1KNpR2yc71uBDKI1Y+ldpbauXH/2P8cnZZUvF6sRKPpS+6N4m2IjDjDpJsA50s= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=FcAo3L1G; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="FcAo3L1G" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62K6mBAM475030; Fri, 20 Mar 2026 14:28:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=jwHEqX TgqnKNZ6kOW8Q1JAtXiLwJcNtyZYMr/JVWy78=; b=FcAo3L1GhRizYnv54t78f1 TaR/pYt6jUZ2UY7FcLC7ksiD9vkTdNl1BJ/4Uoq4FbDswJheJihlGctPdeb1gB8v YPTSHCpWW/7phfuTHY80JzVvpI8L33I2O3dbDuD3xa5lx0KMFJT7eBK9PiiBruT0 HsKTs0ktIQAtGk0GHLaDhTDMeRcT9FY8bY3It269hNdhwWJhdDdzaSwA4NVy3m+4 BrK8osCW0zrwE++k4KX/VFOIE0MW17nX7tUJiNHsSxktwQvyBZYRocJr/t66Xy84 epfkwOdRiryHDANla5DH+XawqzzNVnOBuBLNwP+hbNQn5BDUd2EITFO2p/zNzJew == Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvyauu2kc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 20 Mar 2026 14:28:33 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62KDTNC2028496; Fri, 20 Mar 2026 14:28:32 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwmq1q9wq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 20 Mar 2026 14:28:32 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62KESVjI57082198 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 20 Mar 2026 14:28:31 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EF2F520040; Fri, 20 Mar 2026 14:28:30 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BBBB420043; Fri, 20 Mar 2026 14:28:28 +0000 (GMT) Received: from [9.39.23.72] (unknown [9.39.23.72]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 20 Mar 2026 14:28:28 +0000 (GMT) Message-ID: <786a8ad6-e49e-4ac0-8a1d-514987e24e83@linux.ibm.com> Date: Fri, 20 Mar 2026 19:58:27 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/2] sched/fair: consider hk_mask early in triggering ilb From: Shrikanth Hegde To: Peter Zijlstra Cc: K Prateek Nayak , mingo@kernel.org, vincent.guittot@linaro.org, linux-kernel@vger.kernel.org, juri.lelli@redhat.com, vschneid@redhat.com, tglx@linutronix.de, dietmar.eggemann@arm.com, frederic@kernel.org, longman@redhat.com References: <20260319065314.343932-1-sshegde@linux.ibm.com> <20260319065314.343932-2-sshegde@linux.ibm.com> <6f78a46d-9f12-4e42-ae9b-17f3543136e7@amd.com> <20260320114312.GB3558198@noisy.programming.kicks-ass.net> <8b638b28-9fbf-40f9-8c5a-e6485d9aea2b@linux.ibm.com> Content-Language: en-US In-Reply-To: <8b638b28-9fbf-40f9-8c5a-e6485d9aea2b@linux.ibm.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzIwMDExMyBTYWx0ZWRfX94xDXPZZnCE3 pfe5hmnfuiE6fIMm0uX/Wd90PtB6pXafKn9k2Q0cyuT6+oogPkIDhPuWJRNhs0K5cg28ouhc22N pY89sFjyRMJvTYCGAjllkk58+UfYJa63baQrC3Q/Q7Rl/Id6tXZ5hAZeD8QygAZ6A9fTq6/ke9T ik5ik6lcp928Wp2LlcmBxJQcvAjeMqOW0fpI3WkdR7QFYk1khRFopVm94a6quD2FynpXkYr6LIt IzZg4yx0GVJoPtAykNvp7EihyZch0xNhnzwGRgRtDekQ7xJBFPi5id+E3KHWB0DQ/Bv2esu1YLV kM6IIOcpskv5cNQjiDOtypj1OmwBivGF92lQwEG0i0y8CPqPLeEH7E4zmMaTONtoNBKHaV8OCSW C9FXRLvC5s6DZtvfjxXhcb/tklIxu0K+LEdh8Wo9xKEpLV5tcLcUIv7FW/zBnc65LmJRep2Nt9J WGF91/+DoH9R/1suJrg== X-Authority-Analysis: v=2.4 cv=GIQF0+NK c=1 sm=1 tr=0 ts=69bd5991 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=IkcTkHD0fZMA:10 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=ycNwkIuV15-cEGZrsBUA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: sUutP0wX6duHm6ilmRatxS0d8jev6lSR X-Proofpoint-GUID: sUutP0wX6duHm6ilmRatxS0d8jev6lSR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-20_02,2026-03-19_05,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 impostorscore=0 suspectscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603200113 On 3/20/26 7:42 PM, Shrikanth Hegde wrote: > > > On 3/20/26 5:13 PM, Peter Zijlstra wrote: >> On Fri, Mar 20, 2026 at 02:49:30PM +0530, Shrikanth Hegde wrote: >>> >>> >>> On 3/20/26 9:07 AM, K Prateek Nayak wrote: >>>> Hello Shrikanth, >>>> >>>> On 3/19/2026 12:23 PM, Shrikanth Hegde wrote: >>>>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c >>>>> index b19aeaa51ebc..02cca2c7a98d 100644 >>>>> --- a/kernel/sched/fair.c >>>>> +++ b/kernel/sched/fair.c >>>>> @@ -7392,6 +7392,7 @@ static inline unsigned int >>>>> cfs_h_nr_delayed(struct rq *rq) >>>>>    static DEFINE_PER_CPU(cpumask_var_t, load_balance_mask); >>>>>    static DEFINE_PER_CPU(cpumask_var_t, select_rq_mask); >>>>>    static DEFINE_PER_CPU(cpumask_var_t, should_we_balance_tmpmask); >>>>> +static DEFINE_PER_CPU(cpumask_var_t, kick_ilb_tmpmask); >>>> >>>> nit. We can rename and reuse select_rq_mask. Wakeups happen with IRQs >>>> disabled and kick happens from the hrtimer handler so it should be safe >>>> to reuse that and save some space. >>>> >>>> Thoughts? >>> >>> May be. but it could be a confusing name. sched_tmpmask? >>> >>> We could similar stuff already to load_balance_mask, select_rq_mask. >>> So, i would prefer to keep it separate. >> >> But then we keep growing this ad infinitum. >> >> The more sensible option is to name them after the context and have >> get/put accessors that (for PROVE_LOCKING builds or so) verify the >> context and maybe even 'lock' them to make sure nobody is trying to use >> one for two things at the same time. >> Maybe we can define one cpumask which all irq disabled section could use? i.e for try_to_wake_up and nohz_balancer_kick? Is that what you meant? >> That should make it clearer whats what and improve reuse, no? >> > > We have these: > > deadline.c:static DEFINE_PER_CPU(cpumask_var_t, local_cpu_mask_dl); > ext_idle.c:static DEFINE_PER_CPU(cpumask_var_t, local_idle_cpumask); > ext_idle.c:static DEFINE_PER_CPU(cpumask_var_t, local_llc_idle_cpumask); > ext_idle.c:static DEFINE_PER_CPU(cpumask_var_t, local_numa_idle_cpumask); > fair.c:static DEFINE_PER_CPU(cpumask_var_t, load_balance_mask); > fair.c:static DEFINE_PER_CPU(cpumask_var_t, select_rq_mask); > fair.c:static DEFINE_PER_CPU(cpumask_var_t, should_we_balance_tmpmask); > rt.c:static DEFINE_PER_CPU(cpumask_var_t, local_cpu_mask) > > Here, > 1. load_balance_mask and should_we_balance_tmpmask are used in the same > context >    all it cares is - I need two cpumask to play with. > > 2. local_cpu_mask, local_cpu_mask_dl, select_rq_mask are used in > independent context - all it cares is >    all it cares is - I need one cpumask to play with. > > 3. ext_idle.c: local_* are used the same context. >    all it cares is - I need three cpumask to play with >    technically it is doable with one.(but that's a separate story) > > 1 and 2 are in interrupt disable section, 3 i am not completely sure. > > > I am wondering something like would make sense? > > > DEFINE_PER_CPU(cpumask_var_t, sched_tmp_cpumask1); > DEFINE_PER_CPU(cpumask_var_t, sched_tmp_cpumask2); > DEFINE_PER_CPU(cpumask_var_t, sched_tmp_cpumask3); > > Request for a tmp cpumask with number? > i.e 2 would say sched_request_tmpmask(0) >     1 would sat sched_request_tmpmask(0), and then > sched_request_tmpmask(1) >     3 would say sched_request_tmpmask(0), sched_request_tmpmask(1) and > sched_request_tmpmask(2) > > Do this if interrupts are disabled. > if interrupt are enabled, then maybe do allocation/free instead? > 1 is in softirq. It need not have interrupts disabled. So doing allocation there is too costly.