From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22C4F2D73A6 for ; Thu, 11 Jun 2026 19:19:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781205571; cv=none; b=BRYKzisw/KpaSlDof6P4DZGke91Wt8vzJ8dWyF1M4RSVEBgWAhanMZaG89WSi4is+t6ZKpSjuLcfR70m7YaHYzKhJV3kqi43iydYy4v+63p2wzBw3NgZM1L3t7Q12nkSHTe4uq6oJLcA1nFs4FxXQ/9PZuWSPTudfK0gK1RZFkk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781205571; c=relaxed/simple; bh=Ls18rJmixks2Asp74tpU/nmd406KQEtTOAqM3xoTILk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZFcJ/v2NOfPYDqwgtfoXKrbHU2rqE0TOUUkTTmZFUgzsVa/hm5KjAtxeV5vcC7PZ/GHutwnzchFvOscSDJKdGBc/Qq0oS3VdkVUB2uozD3e2juOu20mpkb0B3JjN0PRu54Htx9DuxXpJIIseObazRPDe7afwG5EDgOmudfMAjzM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=pass smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=JEGV9DMs; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="JEGV9DMs" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=bITd2A29ihHLLIhBSw/vhmASjLxkZ0Nh/MoNmvq6gzU=; b=JEGV9DMsb9YPSA+61MmRAzP7pd EiUkceaof4+vnQf8DLnym/NtjaNo2KHW6+6EmrGqfCbT6+EvjUY10eurLnHlFgSJ0r1rhtziK4uMW k+7Kx6uBuT8lIvOLJ9lvkaTFHQJqD+sQKwJiaSPrcx7/aCsMOLNkenGD1pHLgZR0QTtYZ5x/dVkYi ntY1fQ/2YaA1/g8Fz4gq2sM/KjPZOPalQemA1twNjLK272SC5UpzJUqFxDQateziq0QwBTEPbAk3D 82cHzFLTBf7oAimczWHyI96FaPza5N/hM8w4a23HDCT2JsNxNKg7sDJWMxVpJhlOif6zRQHKPumXs wROhXU7Q==; Received: from 77-249-17-252.cable.dynamic.v4.ziggo.nl ([77.249.17.252] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.99.2 #2 (Red Hat Linux)) id 1wXkvz-00000005aye-3M3T; Thu, 11 Jun 2026 19:19:23 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 6734E301A42; Thu, 11 Jun 2026 21:19:22 +0200 (CEST) Date: Thu, 11 Jun 2026 21:19:22 +0200 From: Peter Zijlstra To: Rik van Riel Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, vschneid@redhat.com Subject: Re: [PATCH 0/9] sched: make WARN_ON under rq->lock deadlock-safe (SCHED_WARN_ON) Message-ID: <20260611191922.GK187714@noisy.programming.kicks-ass.net> References: <20260611021416.910555-1-riel@surriel.com> <20260611074344.GG48970@noisy.programming.kicks-ass.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Thu, Jun 11, 2026 at 12:20:27PM -0400, Rik van Riel wrote: > On Thu, 2026-06-11 at 09:43 +0200, Peter Zijlstra wrote: > > > > Sorry, no. I've said it before and I'll stick with it. Just no. > > > > printk_deferred() is an abomination, it means that if you mess up the > > machine properly you'll *NEVER* see the output. > > > > As per always, printk() is the one that needs fixing, and IIRC they > > were > > very close to getting there. > > Printk to certain console types is always deferred, > by default, because trying to synchronously print > everything to a slow serial console can lead to a > system softlockup panic. > > In fact, this particular lockup is due to the > printk being passed off to a worker thread, and > the kernel deadlocking when the wakeup code > tries to grab the runqueue lock its CPU already > holds. > > You are right that this could be fixed in the > printk code, but the solution there will by > necessity continue to contain some deferring. > > > I suppose the printk code could use something > like an irq work to wake up the printk worker, > and avoid the scheduler deadlock that way? > > I'm not sure that would make things more > reliable, though... The non-atomic consoles will always need a buffer, but atomic consoles can (and should IMO) push out the messages immediately. Anyway, the thing that keeps tripping is that console_sem thing, that should just entirely go away. That thing ends up doing a wakeup from printk() call context, which obviously doesn't work when inside the scheduler locks. So printk should: - stick msg in buffer (lockless) - print to atomic consoles (lockless) - use irq_work to wake console kthreads (lockless) - each kthread then tries to flush buffer to its own non-atomic console in non-atomic context. >From what I understand, we're very close to having this work. The only disagreement between printk people an me is about defaults IIRC. I want to have the serial console default to atomic, they want it to be an option. But whatever, as long as I can specify on the kernel cmdline that my serial should be atomic I'm good. Myself, I almost exclusively run with earlycon serial and force printk to be early_printk() (effectively not using printk at all). This works perfectly fine and is the most reliable thing ever. I can push out characters to the UART from any context. And sure, sometimes its gets a little scrambled, but meh. I have this working with real actual serial, IPMI/serial-over-lan and AMT/serial-over-lan. And if a machine doesn't have serial, its a paperweight ;-) Now, of course I also use trace_printk() a lot, but for those moments when the machine goes down hard, nothing beats serial.