From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8ED7391E51 for ; Thu, 23 Apr 2026 19:12:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776971542; cv=none; b=OqGUn40T67z/1WRL2tTHDutp0mrnMh7HKv8LJS5I0C0QMhjyf/5K5dm++FNhmwaBUb9Vp+EoOmDVuoYoBmcdbK2a6x4zsMsf3JXndB35iV0lsj8HUJ4EOxh2MT6FrDhf6T6ZkYc8sMX1sDApmIvWHlgCtOXZ30JCyipMeRMaqro= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776971542; c=relaxed/simple; bh=Ynuarxho7dbXEjFCU4Qbo+DBCHiq0RRfaHAXB2lk+P0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BvADN1/Nv4ue77MKgJXx5g4DuE2HTEUdgWZZ1sXw9Gp1AUZPBxS8B6YI+bZrWf4eaY9ldZN6bVdGkRbSuejGTSCya4Dsp3njWDdBI8mniz1yR4DMOR4WopucyYoAFf5UoDekQs5CLoAVo6/TC8WuTlWnA8xKVLUKgowMjiT0b5U= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jpiecuch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PEnCKDPS; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jpiecuch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PEnCKDPS" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-43e52dc8a04so6180479f8f.3 for ; Thu, 23 Apr 2026 12:12:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776971539; x=1777576339; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=aVua60xrwR/dQ0PGs66L9Kb497eWff2wlz8NKw9/BJY=; b=PEnCKDPSkX3d1dMzzv15A1fbTatXoHHmcxi66LmD9R1KxmElNUOCO5dY6+a4E9MjpI mqQidYVbg5ygfm1fx7us1yCosMVHmoVkrkBu6C9DyiCTKoBixzhx56lQnCvbfQR68mCX gS3Yk/GYxL8hOzssh9dEtqWPH4uu/K0AMXURtGTFivs+gMp+mRu0AYQgoZBfKRmxuA6y rXduWM7M7aTDU3a2VGo7x54t8OWo6jdIippL0WBvJrGg1VXETVNvasDmoN9+4waSMPPr C7Q16apz6xJau2NnZHr70953roGnBCRzXakku14sl5CrLRx7s8arQ21NI61S98Hl0zx5 5XpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776971539; x=1777576339; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aVua60xrwR/dQ0PGs66L9Kb497eWff2wlz8NKw9/BJY=; b=IvHXvDvssF6HPI5uqRM+HfGMsbuHYqqM2prOky1ATkr0FyeS0kXuaKBVZcV/Uh79oO fpPJqLmenIua5au80Qb0KkROhTA5dMivnMMbotzBvIhXx+ssuvby5d5XauDxNskVabxi VgfG0XODBfXO7CY9QvhwTLzssDbmX5fD33PTGtIzxuTsB5tvMyJdu4kqyZnlhLikid/X slq/f8QeaKpcNzbGbx86I2j5kB9tWeHiBdKbL8t3kBlz/wk9FG40EcQ95Dq5wtsI0LeJ RAV8KJM2UxF3yUS0u6WJHx6lOUPBNLqlulP3Q/aT1PUkVj08T2imwe6eCTHwvBLdJrF6 iR9A== X-Forwarded-Encrypted: i=1; AFNElJ8iA77tHlAFd7C3qTmhHcJhCWFDkYmFa+agj8hlE9iAtpGq898ZHECsdpyKY+kiJxqrm42dzu8o/hmoSLA=@vger.kernel.org X-Gm-Message-State: AOJu0Yyyey9aY9rrY+/540mMqGHfaE3IKBAhWGsZ4teTrG6GeWIZR7v9 wxIxrYO19dmhAko+YfluqNkRM8edZQ7oSM+k9QpuKFPfExmkfMfE7sfQGs+weE9rdhGcPlVI41I mRzFo0iBQLyk4eA== X-Received: from wrpy7.prod.google.com ([2002:adf:f6c7:0:b0:441:2008:76f1]) (user=jpiecuch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:2284:b0:43d:7377:8772 with SMTP id ffacd0b85a97d-43fe3e25982mr44778383f8f.46.1776971539255; Thu, 23 Apr 2026 12:12:19 -0700 (PDT) Date: Thu, 23 Apr 2026 19:12:18 +0000 In-Reply-To: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: aerc 0.21.0-0-g5549850facc2 Message-ID: Subject: Re: SCX_ENQ_IMMED potentially leaving dispatched tasks lingering on local DSQs From: Kuba Piecuch To: Tejun Heo , Kuba Piecuch Cc: Andrea Righi , Changwoo Min , David Vernet , , Content-Type: text/plain; charset="UTF-8" On Thu Apr 23, 2026 at 4:53 PM UTC, Tejun Heo wrote: > Hello, > > On Thu, Apr 23, 2026 at 09:48:11AM +0000, Kuba Piecuch wrote: >> I'm not sure if we should limit ourselves to just remote tasks. >> If we call wakeup_preempt(rq, p, flags) when adding @p to @rq's local DSQ >> regardless of whether @p/@rq is remote, then I think that should cover all >> cases and the patch above wouldn't be needed. > > Sorry, I wasn't clear. I meant adding wakeup_preempt() in > move_remote_task_to_local_dsq() *in addition to* your nr_immed/RETRY_TASK > patch. That combination covers both cases: > > - Local (your stepwise scenario): the task stays on the dispatching CPU, no > migration happens, so there's no insertion-side hook to arm. Your nr_immed > check in do_pick_task_scx() catches it on the RETRY_TASK path. > > - Remote: move_remote_task_to_local_dsq() is SCX's analog of > move_queued_task() and is missing the wakeup_preempt() call that > move_queued_task() does after activate_task(). Once added, > dst_rq->next_class is bumped to &ext_sched_class, and any subsequent > higher-class wakeup on dst_rq routes through wakeup_preempt_scx() which > already has the nr_immed reenqueue. There's another case we need to handle: dispatching to a remote CPU's local DSQ, but without migrating the task. In dispatch_to_local_dsq() this corresponds to rq != src_rq && src_rq == dst_rq. The check for the local case on the remote CPU doesn't cover it because an RT task can swoop in and prevent pick_task_scx() from running (the assumption is that the remote CPU is idle). The check for the remote case doesn't cover it either because we do dispatch_enqueue() directly from dispatch_to_local_dsq(). Then there's the same case, but instead of dispatching we're moving the task from a user DSQ. If my understanding of the code is correct, the path is: scx_dsq_move(p, dsq_id); raw_spin_rq_unlock(this_rq); raw_spin_lock(src_rq); dst_dsq = find_dsq_for_dispatch(dsq_id); move_task_between_dsqs(p, src_dsq, dst_dsq); dst_rq = container_of(dst_dsq, struct rq, scx.local_dsq); task_unlink_from_dsq(p, src_dsq); move_local_task_to_local_dsq(p, src_dsq, dst_rq); I think these two cases show that we don't necessarily have to migrate a task for wakeup_preempt() to be warranted. So to cover all cases (that I'm aware of), we need four checks: * wakeup_preempt() in dispatch_to_local_dsq() in the case of rq != src_rq && src_rq == dst_rq * wakeup_preempt() at the end of move_remote_task_to_local_dsq() * wakeup_preempt() somewhere on the scx_dsq_move() path, but only if we're moving the task to the local DSQ of a remote CPU (?) * nr_immed check before returning RETRY_TASK > > Adding wakeup_preempt() to local_dsq_post_enq() for every insertion would > work too, but I'd rather keep it in sync with how core sched handles > move_queued_task() and only add it where it's actually needed. Does having four separate cases to handle, not all of which involve task migration, change that calculus somewhat? I believe all of these cases will be handled if we add wakeup_preempt() to local_dsq_post_enq(). Thanks, Kuba