From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756240AbcBHVLa (ORCPT ); Mon, 8 Feb 2016 16:11:30 -0500 Received: from smtprelay0179.hostedemail.com ([216.40.44.179]:57264 "EHLO smtprelay.hostedemail.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754655AbcBHVL2 (ORCPT ); Mon, 8 Feb 2016 16:11:28 -0500 X-Session-Marker: 726F737465647440676F6F646D69732E6F7267 X-Spam-Summary: 2,0,0,,d41d8cd98f00b204,rostedt@goodmis.org,:::::::::,RULES_HIT:41:355:379:541:599:800:960:973:981:988:989:1042:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1534:1542:1593:1594:1711:1730:1747:1777:1792:2393:2553:2559:2562:2897:3138:3139:3140:3141:3142:3355:3622:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4470:5007:6119:6261:6691:7875:8531:10004:10400:10848:10967:11026:11232:11658:11914:12291:12296:12517:12519:12555:12740:13095:13161:13229:13846:14096:14097:14659:21080,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:5,LUA_SUMMARY:none X-HE-Tag: mine77_80c3935b43a43 X-Filterd-Recvd-Size: 3787 Date: Mon, 8 Feb 2016 16:11:23 -0500 From: Steven Rostedt To: Denys Vlasenko Cc: linux-kernel@vger.kernel.org, srostedt@redhat.com, Tejun Heo , Peter Hurley Subject: Re: [PATCH] printk: avoid livelock if another CPU printks continuously Message-ID: <20160208161123.2ceb601d@gandalf.local.home> In-Reply-To: <1454963703-20433-1-git-send-email-dvlasenk@redhat.com> References: <1454963703-20433-1-git-send-email-dvlasenk@redhat.com> X-Mailer: Claws Mail 3.13.1 (GTK+ 2.24.29; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 8 Feb 2016 21:35:03 +0100 Denys Vlasenko wrote: > This patch is reported to make affected user's machine survive. Would be nice to have a test case for this. Make a test module to reproduce the issue? > > Signed-off-by: Denys Vlasenko > CC: linux-kernel@vger.kernel.org > CC: srostedt@redhat.com > CC: Steven Rostedt > CC: Tejun Heo > CC: Peter Hurley > --- > kernel/printk/printk.c | 25 +++++++++++++++++++++++++ > 1 file changed, 25 insertions(+) > > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c > index c963ba5..ca4f9d55 100644 > --- a/kernel/printk/printk.c > +++ b/kernel/printk/printk.c > @@ -2235,6 +2235,7 @@ void console_unlock(void) > unsigned long flags; > bool wake_klogd = false; > bool do_cond_resched, retry; > + unsigned cnt; > > if (console_suspended) { > up_console_sem(); > @@ -2257,6 +2258,7 @@ void console_unlock(void) > /* flush buffered message fragment immediately to console */ > console_cont_flush(text, sizeof(text)); > again: > + cnt = 5; > for (;;) { > struct printk_log *msg; > size_t ext_len = 0; > @@ -2284,6 +2286,9 @@ skip: > if (console_seq == log_next_seq) > break; > > + if (--cnt == 0) > + break; /* Someone else printk's like crazy */ > + > msg = log_from_idx(console_idx); > if (msg->flags & LOG_NOCONS) { > /* > @@ -2350,6 +2355,26 @@ skip: > if (retry && console_trylock()) > goto again; > > + if (cnt == 0) { > + /* > + * Other CPU(s) printk like crazy, filling log_buf[]. > + * Try to get rid of the "honor" of servicing their data: > + * give _them_ time to grab console_sem and start working. > + */ > + cnt = 9999; I'll ignore that this looks very hacky. > + while (--cnt != 0) { > + cpu_relax(); > + if (console_seq == log_next_seq) { First, console_seq needs logbuf_lock protection. On some archs, this may hit 9999 every time as the console_seq is most likely in cache and isn't updating. Not to mention the race of another task moving log_next_seq too and this could have been on another CPU changing both console_seq and log_next_seq. Perhaps just save off console_seq and see if it changes at all. > + /* Good, other CPU entered "for(;;)" loop */ > + goto out; > + } > + } > + /* No one seems to be willing to take it... */ > + if (console_trylock()) > + goto again; /* we took it */ Perhaps add a few loops to the taking of the console sem. But again, this just sounds like playing with heuristics, and I hate heuristics. There's gotta be a better solution. -- Steve > + /* Nope, someone else holds console_sem! Good */ > + } > +out: > if (wake_klogd) > wake_up_klogd(); > }