From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6977819995E for ; Wed, 8 Apr 2026 12:19:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.51 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775650779; cv=none; b=RxGCVAl/ltI2fUlyMF1N72pifb4eStLq0HbCma+DH1EIZUCNh5GkDixLQ83QcGb209NeWRi/k4bghnuYYldWtt2Jrg2oaKUDWc/A0319dpMD6KPm5umWsz5esx9cX+19w/KUsrKFh6+128rZt6MlqipAGm6324b1YkC+wyBBH0U= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775650779; c=relaxed/simple; bh=pmb9QeN9jx6NyJYToWRchTvdb7UfSZ22EzX/D81Ku6o=; h=Date:From:To:Cc:Subject:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=d+yq0YN5XHP+i0jfo4G+0/GCOFUEyF+nOFVUFp/32ouFstaGccV8zh1VWWKK0y8MM2JWTMox8etZdm2TZhf7L8BFJRAn+0FQlsn3OG57tC42xPw4HBRh9asiZ2cIiN+9JemVtpTKLXVZH3E14E7PSCKX6S6ehR5b+L4yLPKVbXg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QfxsX2Ys; arc=none smtp.client-ip=209.85.128.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QfxsX2Ys" Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-48896199cbaso56695795e9.1 for ; Wed, 08 Apr 2026 05:19:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775650777; x=1776255577; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=K9fL/OVnhXUuW+CNKCA6bD6e1OkdrxK7ZCqoG06CaUU=; b=QfxsX2YshE9saLgq+UqHcs0Vib4dZJMAij0zDn485OFJxVFOtc3gKvMsx7bTxkrO9h x40TFqnH40uwO1hxrR3bVUoumidruebbcCznI+W6T3/891dd3C+gs9WcMF6/84Ji+QU2 LwYGTFKyTJrvWQGIpccfbdqz0ubXHRr7MMy/IaOYCqHYF/utn3yGVYqPgtqHBvNlm0G0 aoGM7H4ozCRytEAzyBrxjZ+GgQduK6P3or4P5+SREIN3n3c7sydyn8Aly3FBWnkKNP2p yWt97CXlOIqEFatPQ/ScueJh+VinX7Ik6R3JXNafzP3vnQzLK2Y3ONrCasE1/hQEHXua I4EA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775650777; x=1776255577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=K9fL/OVnhXUuW+CNKCA6bD6e1OkdrxK7ZCqoG06CaUU=; b=dfO0Od++46MuJPeTboFE1+IO/i0hiUsPpNocW6ozs3kAie41h8rUhkFCRwByANXxf3 uwXci2Q4bzainpSBry75KU94maSXKUPYMMcIxL2f5K538yC4ZCVikZyZshlYvWFP8QoX 64ZMiHSScIed4ZQr8LfIQlFQ7C2eBNX5zbmELblxnuLisnYGitvbWhxeC0HPpOPezPYk nQTF3/gp4Ext1SxbPEfupx9wRCPxZzr7MejhGwgwoZSx+eTMhJNweRK+vE61cv5Y2XIl X5cMMTF+atfEIi5nKv0mlQ5CqAA79AORagRKUPHmGgDT87bUPPqJY/CvRHzwW8Wu2fLg Eyeg== X-Forwarded-Encrypted: i=1; AJvYcCV2zpRKtZjpaK0FRdsLvs/ohfpAI9u/QfD8HuCzaLteRn7smHt8E/cvvy7kYfrVohDnhB1U/TDhpvfBKX4=@vger.kernel.org X-Gm-Message-State: AOJu0Yw1cGwT0zOfYOAHLWL+3p+u4zi8kuH05woI+I56mzRIfEdqzr8D P5aiPl/B6vdTD6//6sAwNtJnKUkw1lUb0CeZBRarK2FE7JaJrR7AtU9D X-Gm-Gg: AeBDieu5fxd1srLDkXsN0lKoJIhUQnKKBV12NukEgOt2jbSmRDme1KnKCvc3xEbVJ12 tMU0Ynl3tbSRtJj2jDunW/ZY1He7+mFLNRl0eurMUWf3ASSBDWwGs18WHqUUgfBcwOZZ0cAFkFt CBGBNDJwC/3fI2H6Z16sU7tw60po9v4RS19A0Mn6hVfGfjwA+g280oWHK1mPaBFJc/rY6OtuN8n 4Z2nbu9wPYvg5LIGHAMGxhZHbPWbTCm4g5auB5404Rk/JqwNV4FJIrWvqC7yYMQgPUJksbH7Mpv 6ZmWRukRw6NPkC59Vdb29gMwMIbadBfV1jts/PTy/0YNaw87XI5Ar6z3AH/FKDCa280bfR/1Q+1 /OVfEeQwoD/6SonucYkzAYg/XxCxwx9eBJ5qqotB7BJ5Z7V48qVVRwQW3rLp4+SAW9X1i3pCf4R mg7zQ3H75QaMlseJY0XHKzzpddPyj7AUp1BYNpYkXbnW2DuDsc5S/GZIgtXV6F1V6P X-Received: by 2002:a05:600c:820e:b0:485:40c6:f507 with SMTP id 5b1f17b1804b1-488997ee0bdmr295124905e9.30.1775650776604; Wed, 08 Apr 2026 05:19:36 -0700 (PDT) Received: from pumpkin (82-69-66-36.dsl.in-addr.zen.co.uk. [82.69.66.36]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43d1e4e5890sm57170605f8f.31.2026.04.08.05.19.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2026 05:19:35 -0700 (PDT) Date: Wed, 8 Apr 2026 13:19:30 +0100 From: David Laight To: Xuewen Yan Cc: Peter Zijlstra , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Benjamin Segall , Mel Gorman , Valentin Schneider , linux-kernel , "=?UTF-8?B?546L56eR?= (Ke Wang)" , Xuewen Yan , "hongyu.jin@unisoc.com" , guohua.yan@unisoc.com, John Stultz Subject: Re: [Question] Sched: Severe scheduling latency (>10s) observed on kernel 6.12 with specific workload Message-ID: <20260408131930.28c1e80a@pumpkin> In-Reply-To: References: X-Mailer: Claws Mail 4.1.1 (GTK 3.24.38; arm-unknown-linux-gnueabihf) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, 1 Apr 2026 10:32:30 +0800 Xuewen Yan wrote: > Dear Linux maintainers and reviewers, >=20 > I am writing to report a severe scheduling latency issue we recently > discovered on Linux Kernel 6.12. >=20 > Issue Description >=20 > We observed that when running a specific background workload pattern, > certain tasks experience excessive scheduling latency. The delay from > the runnable state to running on the CPU exceeds 10 seconds, and in > extreme cases, it reaches up to 100 seconds. Have you managed to get a low priority process spinning in kernel on the cpu the RT process last ran on? The RT process will always run on the same cpu it ran on provided in is higher priority than the current process running on that cpu. ftrace logging scheduler events is you 'friend' here. David=20 >=20 > Environment Details >=20 > Kernel Version: 6.12.58-android16-6-g3835fd28159d-ab000018-4k > Architecture: [ ARM64] > Hardware: T7300 > Config: gki_defconfig >=20 > RT-app=E2=80=98s workload Pattern: >=20 > { > "tasks" : { > "t0" : { > "instance" : 40, > "priority" : 0, > "cpus" : [ 0, 1, 2, 3 ], > "taskgroup" : "/background", > "loop" : -1, > "run" : 200, > "sleep" : 50 > } > } > } >=20 > And we have applied the following patchs: >=20 > https://lore.kernel.org/all/20251216111321.966709786@linuxfoundation.org/ > https://lore.kernel.org/all/20260106170509.413636243@linuxfoundation.org/ > https://lore.kernel.org/all/20260323134533.805879358@linuxfoundation.org/ >=20 >=20 > Could you please advise if there are known changes in the eevdf in > 6.12 that might affect this specific workload pattern? >=20 > Thanks! >=20 > BR > --- > xuewen