From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 563C6438FEA for ; Tue, 16 Jun 2026 10:56:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.52 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781607383; cv=none; b=hlN+ahJtLuNEK0Lsz93NilncrBmszOUxFcXsNkHNUSgH/mqGUeEBU50pKu4ipc8IvaL4NGAgD7/mupDtHAM+yHQighbgpiGRwZIUSSzt36U0EGjkIhtV5z5tKBN1R9HEBAlD9EtL5ZRJc2bK4TclolKQEFyBxrFDumF6mDst5o4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781607383; c=relaxed/simple; bh=9AH6QMSRryDOTfJRxGukJID4bDV6I7wkQ4xXWH3c1ZU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=qmjRq3d+k1R1ciRhi5HFw8IViRJqeLWZfQ7nBxDeB9WfQMnKc9cMnYFkmMqbSdXzWx1j4R1etbVv6eRTpFuXjmPBk3TQ0Rr/eimwZ8ZfDFvAORmKISe9IC7ZbAb7jlPmyBdourufqJ2WsoGd+Uexhtu1ddD3WGqVg/9B35k0EPU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=readmodwrite.com; spf=none smtp.mailfrom=readmodwrite.com; dkim=pass (2048-bit key) header.d=readmodwrite-com.20251104.gappssmtp.com header.i=@readmodwrite-com.20251104.gappssmtp.com header.b=DMAGol3N; arc=none smtp.client-ip=209.85.208.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=readmodwrite.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=readmodwrite.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=readmodwrite-com.20251104.gappssmtp.com header.i=@readmodwrite-com.20251104.gappssmtp.com header.b="DMAGol3N" Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-691c5776f35so7134421a12.3 for ; Tue, 16 Jun 2026 03:56:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=readmodwrite-com.20251104.gappssmtp.com; s=20251104; t=1781607381; x=1782212181; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=JFoi32RPtT16jhLdDNPUhdeEQXGpdyQg1sRcbSQgvlw=; b=DMAGol3Nuovk0x0kyGIuwTUPX6Ho3fi7xC4fSyhpcQd6U6yvXrqV2q2234mr4B9rxN N78cL8k06yqj3ej5sHghl9Xnixm2uRJOnioQSFvW3htV7ULCLQHY9+Rnil2/Z5gOjkrv Kc7Zchlcr1CEuwMa+2xFZ277uc2NohrdDIvHfO4OniHWVcaIKDGNvH/5hrJ16UfoSYGk W2YRJFpNv6XRlbQEYS+RmQktmNxoNReuEbYXHdchvquw0vpz65Tx6qGacz+e5Vuprg6n VsHNWwc1bvX+KAz9FxOO56tyGGUOqmNjM5V3PK7BnXMuuz6cyWxhdTv/eboIAR/SK/yg 2j5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781607381; x=1782212181; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JFoi32RPtT16jhLdDNPUhdeEQXGpdyQg1sRcbSQgvlw=; b=XCsc4DeSOvLU2TY3EEFuHBU3Ydnru3C8eOqYUo+tAPaVjEQQ89mw4CqLz7urfLA5lz Iubq82FQmm1oBeKYziM3Dxhlp1IpDbopPuZGobWrF2dsrYzWUffNWjOqrlGIQnEd6UsU b471a6x0p5moYPiCWhL5GRh37Gw0VsdYsT8P8bHhpbMMNkIr86ZhKkDmE2CViWcPZ98W G9boOLq3pepNtI0wPqOAHKzVocKmgIp4AEYnIF7c6KO4N0UaSEJiHIXA/cTcHEBP2N6m xE2VcKMHkmaxUxVHH237DAvWIj0W7E8PJfSK1NupB0cOhxgvI5VG9Z+C8+NwSJYWQ+06 ayZw== X-Forwarded-Encrypted: i=1; AFNElJ81XrN+q11BlYjZFTQFiKhGhcJ9USeesp/+84rxq9dD5VyUr9ZmSdSEt6lbSKAqEOiMKax1hNHAZwr0pYg=@vger.kernel.org X-Gm-Message-State: AOJu0YyiYqc07AaYfoH7aRxzxBA2S2K1rXFT0iXP9vq3W40GjvaxF/2A 2HpRwJY7oDlmEDwMGgLLsZw9IeHS4tM7/HZq/ClFasANZL6MwqR6c7OTiTsxn6Ri3Ck= X-Gm-Gg: Acq92OGHtnOhMhet7AXTNQHcbNg9H2uhIJGqwnJWXchpWNTR4factFXLMwr8Yv9KMgM qw4vmHU292O7605C+KzS9CDarwLJS9uH/F3p+JQtOZcGwdV716Cjs64/dRqn7n8nnaS+gQqil3V aH2bqJsHcHUElo5c7xwjvGeJlJ00ZWAfVTBY8MeZ1P/ezi5Efw3Z6jm+r1mgUY9oimMiFrzCGdm 3qvwY5QwKzmNgTVG9XS5A5d3WRGTsCKiO5TGNhzBF3xh/52bD/2sQcmmBTbm6vzRpakxb+nNp5s R++HdcinGzm8ZIv5QbxHhLd8dlDEUFytv9QSkcXXRSP1KG+LuEhDzcAJEBxY/NTiMdGPiKHsnZE NunzPpeHc/+52aT8rFwUN9Cdu8M/UxAhF2D4EtHqEbBW+hXuTg5ApwHeA+0hkA3R98BavJh4WGn Jv9CK4MxBXEg== X-Received: by 2002:a17:906:9c86:b0:bee:280b:8be5 with SMTP id a640c23a62f3a-bff4a10c535mr644235466b.21.1781607380561; Tue, 16 Jun 2026 03:56:20 -0700 (PDT) Received: from localhost ([2a09:bac6:37a8:1cdc::2e0:a5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-bfdb442083esm629814866b.3.2026.06.16.03.56.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Jun 2026 03:56:19 -0700 (PDT) Date: Tue, 16 Jun 2026 11:56:19 +0100 From: Matt Fleming To: "Paul E. McKenney" Cc: Tejun Heo , Andrea Righi , sched-ext@lists.linux.dev, linux-kernel@vger.kernel.org, kernel-team@cloudflare.com Subject: Re: sched_ext/lavd hard lockup in old call_rcu_tasks_generic needadjust path Message-ID: References: <20260609104733.1184001-1-mfleming@cloudflare.com> <2179fc11-3bd5-4d2e-9ad8-609e3356ec09@paulmck-laptop> <1ad83f53-7897-40bc-a503-5a71031aa27f@paulmck-laptop> <3bbd4298-eb01-4784-bee5-caab3d3647b9@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3bbd4298-eb01-4784-bee5-caab3d3647b9@paulmck-laptop> On Fri, Jun 12, 2026 at 07:00:31AM -0700, Paul E. McKenney wrote: > > Huh. Looks like I did not implement RCU Tasks Trace in terms of SRCU > any time too soon. But there is also RCU Tasks and RCU Tasks Rude. > > If we don't backport the SRCU patches, then the obvious alternative is > for call_rcu_tasks*() to defer to IRQ work when invoked with interrupts > disabled. Or is there a better way? What about something like this? ----8<---- >From 6b2dc5002f3413f4f89eb1735259d065b7003a52 Mon Sep 17 00:00:00 2001 From: Matt Fleming Date: Mon, 15 Jun 2026 11:19:43 +0100 Subject: [PATCH] rcu-tasks: Defer callback queue adjustment to irq_work call_rcu_tasks_generic() can run from BPF task-storage teardown while sched_ext still holds rq->lock. The RCU Tasks kthread can concurrently hold cbs_gbl_lock while printing under it, then wake a task on the same rq while the caller waits for cbs_gbl_lock. Queue the adjustment through irq_work instead. This keeps callback enqueueing synchronous while moving cbs_gbl_lock acquisition out of the caller context. Signed-off-by: Matt Fleming --- kernel/rcu/tasks.h | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 2dc044fd126e..92aead9fc200 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -104,6 +104,7 @@ struct rcu_tasks { unsigned long n_ipis; unsigned long n_ipis_fails; struct task_struct *kthread_ptr; + struct irq_work cbs_adjust_irq_work; unsigned long lazy_jiffies; rcu_tasks_gp_func_t gp_func; pregp_func_t pregp_func; @@ -129,6 +130,7 @@ struct rcu_tasks { }; static void call_rcu_tasks_iw_wakeup(struct irq_work *iwp); +static void call_rcu_tasks_iw_adjust(struct irq_work *iwp); #define DEFINE_RCU_TASKS(rt_name, gp, call, n) \ static DEFINE_PER_CPU(struct rcu_tasks_percpu, rt_name ## __percpu) = { \ @@ -144,6 +146,7 @@ static struct rcu_tasks rt_name = \ .call_func = call, \ .wait_state = TASK_UNINTERRUPTIBLE, \ .rtpcpu = &rt_name ## __percpu, \ + .cbs_adjust_irq_work = IRQ_WORK_INIT_HARD(call_rcu_tasks_iw_adjust), \ .lazy_jiffies = DIV_ROUND_UP(HZ, 4), \ .name = n, \ .percpu_enqueue_shift = order_base_2(CONFIG_NR_CPUS), \ @@ -342,6 +345,24 @@ static void call_rcu_tasks_iw_wakeup(struct irq_work *iwp) rcuwait_wake_up(&rtp->cbs_wait); } +static void call_rcu_tasks_iw_adjust(struct irq_work *iwp) +{ + unsigned long flags; + bool expanded = false; + struct rcu_tasks *rtp = container_of(iwp, struct rcu_tasks, cbs_adjust_irq_work); + + raw_spin_lock_irqsave(&rtp->cbs_gbl_lock, flags); + if (rtp->percpu_enqueue_lim != rcu_task_cpu_ids) { + WRITE_ONCE(rtp->percpu_enqueue_shift, 0); + WRITE_ONCE(rtp->percpu_dequeue_lim, rcu_task_cpu_ids); + smp_store_release(&rtp->percpu_enqueue_lim, rcu_task_cpu_ids); + expanded = true; + } + raw_spin_unlock_irqrestore(&rtp->cbs_gbl_lock, flags); + if (expanded) + pr_info("Switching %s to per-CPU callback queuing.\n", rtp->name); +} + // Enqueue a callback for the specified flavor of Tasks RCU. static void call_rcu_tasks_generic(struct rcu_head *rhp, rcu_callback_t func, struct rcu_tasks *rtp) @@ -389,16 +410,8 @@ static void call_rcu_tasks_generic(struct rcu_head *rhp, rcu_callback_t func, rtpcp->urgent_gp = 3; rcu_segcblist_enqueue(&rtpcp->cblist, rhp); raw_spin_unlock_irqrestore_rcu_node(rtpcp, flags); - if (unlikely(needadjust)) { - raw_spin_lock_irqsave(&rtp->cbs_gbl_lock, flags); - if (rtp->percpu_enqueue_lim != rcu_task_cpu_ids) { - WRITE_ONCE(rtp->percpu_enqueue_shift, 0); - WRITE_ONCE(rtp->percpu_dequeue_lim, rcu_task_cpu_ids); - smp_store_release(&rtp->percpu_enqueue_lim, rcu_task_cpu_ids); - pr_info("Switching %s to per-CPU callback queuing.\n", rtp->name); - } - raw_spin_unlock_irqrestore(&rtp->cbs_gbl_lock, flags); - } + if (unlikely(needadjust)) + irq_work_queue(&rtp->cbs_adjust_irq_work); rcu_read_unlock(); /* We can't create the thread unless interrupts are enabled. */ if (needwake && READ_ONCE(rtp->kthread_ptr)) -- 2.43.0