All of lore.kernel.org
 help / color / mirror / Atom feed
From: Tejun Heo <tj@kernel.org>
To: Andrea Righi <arighi@nvidia.com>
Cc: David Vernet <void@manifault.com>,
	Changwoo Min <changwoo@igalia.com>,
	sched-ext@lists.linux.dev, Emil Tsalapatis <emil@etsalapatis.com>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 sched_ext/for-7.3 25/36] sched_ext: Add the SCX_CAP_ENQ_IMMED cap
Date: Wed, 8 Jul 2026 08:20:54 -1000	[thread overview]
Message-ID: <ak6VBhh-2dGkux2v@slm.duckdns.org> (raw)
In-Reply-To: <ak5LXNSj0buDiPMz@gpd4>

Hello, Andrea.

On Wed, Jul 08, 2026 at 03:06:36PM +0200, Andrea Righi wrote:
...
> I bisected it and this was to be the offending commit.
> 
> IIUC, what's happening is that qmap grants the child CPUs 2–3, but race-sched
> stressor can pin its tasks to CPU 1. Since the child has no enqueue capability
> there, the tasks repeatedly land on the reject DSQ and eventually trigger the
> RCU stall.
> 
> Maybe we need a forced fallback when a task's affinity mask has no active CPU on
> which its scheduler holds SCX_CAP_BASE?
> 
> Checking this only during sched_setaffinity() won't work because capabilities
> can be revoked after the affinity change succeeds. What do you think?

Yeah, I should have mentioned in the head message but there are still a few
missing pieces for full support. Two big ones:

- cgroup migrations don't move the tasks across schedulers as they should.

- No fallback mechanism when a sub-sched doesn't have any CPUs for some of
  its tasks.

So, after this series, things mostly work as long as tasks don't move across
cgroup boundaries and tasks in sub-scheds aren't affined to the cpus tha the
sub-sched doesn't have access to. Will update the head message.

I haven't decided on the details yet but it's most likely going to be
reserving some time slice off from the root sched to run the tasks w/o cpu
access.

Thanks.

-- 
tejun

  reply	other threads:[~2026-07-08 18:20 UTC|newest]

Thread overview: 67+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-07  0:11 [PATCHSET v3 sched_ext/for-7.3] sched_ext: Capability-based CPU delegation for sub-schedulers Tejun Heo
2026-07-07  0:11 ` [PATCH v3 sched_ext/for-7.3 01/36] sched_ext: Add SCX_CALL_CID_OP_TASK() for cid-form op dispatch Tejun Heo
2026-07-07  0:11 ` [PATCH v3 sched_ext/for-7.3 02/36] sched_ext: Rename extra_enq_flags to remote_activate_enq_flags Tejun Heo
2026-07-07  0:11 ` [PATCH v3 sched_ext/for-7.3 03/36] sched_ext: Fix premature ops->priv publication in scx_alloc_and_add_sched() Tejun Heo
2026-07-07  0:11 ` [PATCH v3 sched_ext/for-7.3 04/36] tools/sched_ext: scx - Fix cmask_subset(), cmask_equal() and cmask_weight() Tejun Heo
2026-07-07  0:11 ` [PATCH v3 sched_ext/for-7.3 05/36] sched_ext: Use READ_ONCE/WRITE_ONCE in cmask word ops and drop _RACY variants Tejun Heo
2026-07-07  0:11 ` [PATCH v3 sched_ext/for-7.3 06/36] tools/sched_ext: scx_qmap - Use bare u64/u32/s32 integer types Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 07/36] sched_ext: Reject direct slice and dsq_vtime writes for cid-form schedulers Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 08/36] sched_ext: Make scx_bpf_kick_cid() return void Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 09/36] sched_ext: Make the kick machinery per-sched Tejun Heo
2026-07-07  0:33   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 10/36] sched_ext: Add ops.init_cids() to finalize the cid layout before init Tejun Heo
2026-07-07  0:36   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 11/36] sched_ext: Add CID sharding Tejun Heo
2026-07-07  0:30   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 12/36] sched_ext: Add shard boundaries to scx_bpf_cid_override() Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 13/36] sched_ext: Defer scx_sched kobj sysfs add into the enable workfns Tejun Heo
2026-07-07  0:31   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 14/36] sched_ext: Add per-shard scx_sched storage scaffolding Tejun Heo
2026-07-07  0:28   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 15/36] sched_ext: Add scx_cmask_ref for validated arena cmask access Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 16/36] sched_ext: Build the set_cmask scratch from trusted geometry Tejun Heo
2026-07-07  0:30   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 17/36] sched_ext: RCU-protect the sub-sched tree's children/sibling lists Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 18/36] sched_ext: Add scx_skip_subtree_pre() Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 19/36] sched_ext: Add per-shard cap delegation for sub-schedulers Tejun Heo
2026-07-07  0:39   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 20/36] sched_ext: Add coalescing sub_caps_updated() notifier " Tejun Heo
2026-07-07  0:33   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 21/36] sched_ext: Maintain per-cpu effective cap copies for single-read checks Tejun Heo
2026-07-07  0:38   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 22/36] sched_ext: Add sub_ecaps_updated() effective-cap change notifier Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 23/36] sched_ext: Generalize local-DSQ handling to rq-owned DSQs Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 24/36] sched_ext: Add reject DSQ for cap-rejected dispatches Tejun Heo
2026-07-07  0:38   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 25/36] sched_ext: Add the SCX_CAP_ENQ_IMMED cap Tejun Heo
2026-07-07  0:32   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-08 13:06   ` Andrea Righi
2026-07-08 18:20     ` Tejun Heo [this message]
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 26/36] sched_ext: Assign a unique id to each scheduler instance Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 27/36] sched_ext: Route task slice writes through set_task_slice() Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 28/36] sched_ext: Tie cpu occupancy to SCX_CAP_BASE through the task slice Tejun Heo
2026-07-07  0:38   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 29/36] sched_ext: Add the SCX_CAP_ENQ cap Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 30/36] sched_ext: Gate kicks on SCX_CAP_BASE and preemption on SCX_CAP_PREEMPT Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 31/36] sched_ext: Authorize remote-move inserts against the placing scheduler Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 32/36] sched_ext: Route ops.update_idle() to sub-schedulers and re-notify owed scheds Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 33/36] sched_ext: Replay ecaps notifications suppressed by bypass Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 34/36] sched_ext: Add scx_bpf_sub_kill() to evict a child sub-scheduler Tejun Heo
2026-07-07  0:42   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 35/36] tools/sched_ext: scx_qmap - Expand hierarchical sub-scheduling Tejun Heo
2026-07-07  0:41   ` sashiko-bot
2026-07-08 19:09     ` Tejun Heo
2026-07-07  0:12 ` [PATCH v3 sched_ext/for-7.3 36/36] tools/sched_ext: scx_qmap - Add sub-sched cap fault injection Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ak6VBhh-2dGkux2v@slm.duckdns.org \
    --to=tj@kernel.org \
    --cc=arighi@nvidia.com \
    --cc=changwoo@igalia.com \
    --cc=emil@etsalapatis.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=sched-ext@lists.linux.dev \
    --cc=void@manifault.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.