From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S966134AbdLSBD7 (ORCPT ); Mon, 18 Dec 2017 20:03:59 -0500 Received: from mail.kernel.org ([198.145.29.99]:56244 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935486AbdLSBDz (ORCPT ); Mon, 18 Dec 2017 20:03:55 -0500 DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5654F20740 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=goodmis.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=rostedt@goodmis.org Date: Mon, 18 Dec 2017 20:03:51 -0500 From: Steven Rostedt To: Sergey Senozhatsky Cc: Tejun Heo , Sergey Senozhatsky , Petr Mladek , Jan Kara , Andrew Morton , Peter Zijlstra , Rafael Wysocki , Pavel Machek , Tetsuo Handa , linux-kernel@vger.kernel.org Subject: Re: [RFC][PATCHv6 00/12] printk: introduce printing kernel thread Message-ID: <20171218200351.09164cdd@gandalf.local.home> In-Reply-To: <20171219005248.GA8892@jagdpanzerIV> References: <20171204134825.7822-1-sergey.senozhatsky@gmail.com> <20171214142709.trgl76hbcdwaczzd@pathway.suse.cz> <20171214152551.GY3919388@devbig577.frc2.facebook.com> <20171214125506.52a7e5fa@gandalf.local.home> <20171214181153.GZ3919388@devbig577.frc2.facebook.com> <20171215021024.GA11199@jagdpanzerIV> <20171214221831.3ead0298@vmware.local.home> <20171215050607.GC11199@jagdpanzerIV> <20171215101957.5d4a1004@gandalf.local.home> <20171219005248.GA8892@jagdpanzerIV> X-Mailer: Claws Mail 3.14.0 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 19 Dec 2017 09:52:48 +0900 Sergey Senozhatsky wrote: > > The case here, you are talking about a CPU doing console_lock() from a > > non printk() case. Which is what I was asking about how often this > > happens. > > I'd say often enough. but the point I was trying to make is that we can > have non-atomic CPUs which can do the print out, instead of "sharing the > load" between atomic CPUs. We don't even know if sharing between "atomic" and "non-atomic" is an issue. Anything that does a printk() in an atomic location, is going to have latency to begin with. > > > As for why there's no handoff. Does the non printk() > > console_lock/unlock ever happen from a critical location? I don't think > > it does (but I haven't checked). Then it is the perfect candidate to do > > all the printing. > > that's right. that is the point I was trying making. we can have better > candidates to do all the printing. Sure, but we don't even know if we have to. A problem scenario hasn't come up that wasn't due to the current implementation (which my patch changes). > I did tests yesterday, traces are available. I can't conclude that > the patch fixes the unfairness of printk(). It doesn't fix the "unfairness" it fixes the unboundedness of printk. That is what has been triggering all the issues from before. > consider the following case > > we have console_lock() from non-atomic context. console_sem owner is > getting preempted, under console_sem. which is totally possible and > happens a lot. in the mean time we have OOM, which can print a lot of > info. by the time console_sem returns back to TASK_RUNNING logbuf > contains some number of pending messages [lets say 10 seconds worth > of printing]. console owner goes to console_unlock(). accidentally > we have printk from IRQ on CPUz. console_owner hands over printing > duty to CPUz. so now we have to print 10 seconds worth of OOM messages > from irq. Yes that can happen. But printk's from irq context is not nice to have either, and should only happen when things are going wrong to begin with. > > > > CPU0 CPU1 ~ CPUx CPUz > > console_lock > > << preempted >> > > > OOM OOM printouts, lots > of OOM traces, etc. > > OOM end [progress done]. > > << back to RUNNING >> > > console_unlock() > > for (;;) > sets console_owner > call_console_drivers() IRQ > printk > sees console_owner > sets console_waiter > > clears console_owner > sees console_waier > handoff > for (;;) { > call_console_drivers() > ??? lockup > } > up() > > this is how we down() from non-atomic and up() from atomic [if we make > it to up(). we might end up in NMI panic]. this scenario is totally possible, The printk buffer needs to be very big, and bad things have to happen first. This is a theoretical scenario, and I'd like to see it happen in the real world before we try to fix it. My patch should make printk behave *MUCH BETTER* than it currently does. If you are worried about NMI panics, then we could add a touch nmi within the printk loop. > isn't it? the optimistic expectation here is that some other printk() from > non-atomic CPU will jump in and take over printing from atomic CPUz. but I > don't see why we are counting on it. I don't see why we even care. Placing a printk in an atomic context is a problem to begin with, and should only happen if there's issues in the system. -- Steve