From mboxrd@z Thu Jan 1 00:00:00 1970 From: Petr Mladek Subject: Re: [PATCH 19/30] panic: Add the panic hypervisor notifier list Date: Wed, 18 May 2022 09:33:57 +0200 Message-ID: References: <20220427224924.592546-1-gpiccoli@igalia.com> <20220427224924.592546-20-gpiccoli@igalia.com> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1652859243; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YR5rOhmSURb4d/IKXy5VZg8z1ImD01Us4xBpVFtyoqU=; b=apiSUoAwUbqeAccPNQ19K+RPeAfjpiC+Kw09kP9gYD08Bjqn9sy6l8W4iwyq6IQeI6Aplp +YIfJeRJqbcdQptzveOcUb8Q0R7nuJcbMiyUrGxQWxZdDf/W1Ubmr612J8Zmj7lPwrkx5l wG1cTFsW35nAwzkY9hs+1GgrMVmINpw= Content-Disposition: inline In-Reply-To: List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: "Guilherme G. Piccoli" Cc: Evan Green , David Gow , Julius Werner , Scott Branden , bcm-kernel-feedback-list-dY08KVG/lbpWk0Htik3J/w@public.gmane.org, Sebastian Reichel , Linux PM , Florian Fainelli , Andrew Morton , bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, kexec-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, LKML , linuxppc-dev-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org, linux-alpha-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-arm Mailing List , linux-edac-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-hyperv-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-leds-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-mips-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-parisc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-remoteproc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-s390-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-tegra-u79uwXL29TY76Z2rM5mHXA@public.gmane.org On Tue 2022-05-17 13:37:58, Guilherme G. Piccoli wrote: > On 17/05/2022 10:28, Petr Mladek wrote: > > [...] > >>> Disagree here. I'm looping Google maintainers, so they can comment. > >>> (CCed Evan, David, Julius) > >>> > >>> This notifier is clearly a hypervisor notification mechanism. I've fixed > >>> a locking stuff there (in previous patch), I feel it's low-risk but even > >>> if it's mid-risk, the class of such callback remains a perfect fit with > >>> the hypervisor list IMHO. > >> > >> This logs a panic to our "eventlog", a tiny logging area in SPI flash > >> for critical and power-related events. In some cases this ends up > >> being the only clue we get in a Chromebook feedback report that a > >> panic occurred, so from my perspective moving it to the front of the > >> line seems like a good idea. > > > > IMHO, this would really better fit into the pre-reboot notifier list: > > > > + the callback stores the log so it is similar to kmsg_dump() > > or console_flush_on_panic() > > > > + the callback should be proceed after "info" notifiers > > that might add some other useful information. > > > > Honestly, I am not sure what exactly hypervisor callbacks do. But I > > think that they do not try to extract the kernel log because they > > would need to handle the internal format. > > > > I guess the main point in your response is : "I am not sure what exactly > hypervisor callbacks do". We need to be sure about the semantics of such > list, and agree on that. > > So, my opinion about this first list, that we call "hypervisor list", > is: it contains callbacks that > > (1) should run early, preferably before kdump (or even if kdump isn't > set, should run ASAP); > > (2) these callbacks perform some communication with an abstraction that > runs "below" the kernel, like a firmware or hypervisor. Classic example: > pvpanic, that communicates with VMM (usually qemu) and allow such VMM to > snapshot the full guest memory, for example. > > (3) Should be low-risk. What defines risk is the level of reliability of > subsequent operations - if the callback have 50% of chance of "bricking" > the system totally and prevent kdump / kmsg_dump() / reboot , this is > high risk one for example. > > Some good fits IMO: pvpanic, sstate_panic_event() [sparc], fadump in > powerpc, etc. > > So, this is a good case for the Google notifier as well - it's not > collecting data like the dmesg (hence your second bullet seems to not > apply here, info notifiers won't add info to be collected by gsmi). It > is a firmware/hypervisor/whatever-gsmi-is notification mechanism, that > tells such "lower" abstraction a panic occurred. It seems low risk and > we want it to run ASAP, if possible. " > >> This logs a panic to our "eventlog", a tiny logging area in SPI flash > >> for critical and power-related events. In some cases this ends up I see. I somehow assumed that it was about the kernel log because Evans wrote: "This logs a panic to our "eventlog", a tiny logging area in SPI flash for critical and power-related events. In some cases this ends up" Anyway, I would distinguish it the following way. + If the notifier is preserving kernel log then it should be ideally treated as kmsg_dump(). + It the notifier is saving another debugging data then it better fits into the "hypervisor" notifier list. Regarding the reliability. From my POV, any panic notifier enabled in a generic kernel should be reliable with more than 99,9%. Otherwise, they should not be in the notifier list at all. An exception would be a platform-specific notifier that is called only on some specific platform and developers maintaining this platform agree on this. The value "99,9%" is arbitrary. I am not sure if it is realistic even in the other code, for example, console_flush_on_panic() or emergency_restart(). I just want to point out that the border should be rather high. Otherwise we would back in the situation where people would want to disable particular notifiers. Best Regards, Petr From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8F94C433EF for ; Wed, 18 May 2022 07:34:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232180AbiERHeK (ORCPT ); Wed, 18 May 2022 03:34:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232127AbiERHeH (ORCPT ); Wed, 18 May 2022 03:34:07 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D294CC5E55; Wed, 18 May 2022 00:34:04 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 8990E1F9A4; Wed, 18 May 2022 07:34:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1652859243; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YR5rOhmSURb4d/IKXy5VZg8z1ImD01Us4xBpVFtyoqU=; b=apiSUoAwUbqeAccPNQ19K+RPeAfjpiC+Kw09kP9gYD08Bjqn9sy6l8W4iwyq6IQeI6Aplp +YIfJeRJqbcdQptzveOcUb8Q0R7nuJcbMiyUrGxQWxZdDf/W1Ubmr612J8Zmj7lPwrkx5l wG1cTFsW35nAwzkY9hs+1GgrMVmINpw= Received: from suse.cz (unknown [10.100.201.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id AA6042C142; Wed, 18 May 2022 07:33:59 +0000 (UTC) Date: Wed, 18 May 2022 09:33:57 +0200 From: Petr Mladek To: "Guilherme G. Piccoli" Cc: Evan Green , David Gow , Julius Werner , Scott Branden , bcm-kernel-feedback-list@broadcom.com, Sebastian Reichel , Linux PM , Florian Fainelli , Andrew Morton , bhe@redhat.com, kexec@lists.infradead.org, LKML , linuxppc-dev@lists.ozlabs.org, linux-alpha@vger.kernel.org, linux-arm Mailing List , linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-leds@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, linux-tegra@vger.kernel.org, linux-um@lists.infradead.org, linux-xtensa@linux-xtensa.org, netdev@vger.kernel.org, openipmi-developer@lists.sourceforge.net, rcu@vger.kernel.org, sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org, x86@kernel.org, kernel-dev@igalia.com, kernel@gpiccoli.net, halves@canonical.com, fabiomirmar@gmail.com, alejandro.j.jimenez@oracle.com, Andy Shevchenko , Arnd Bergmann , Borislav Petkov , Jonathan Corbet , d.hatayama@jp.fujitsu.com, dave.hansen@linux.intel.com, dyoung@redhat.com, feng.tang@intel.com, Greg Kroah-Hartman , mikelley@microsoft.com, hidehiro.kawai.ez@hitachi.com, jgross@suse.com, john.ogness@linutronix.de, Kees Cook , luto@kernel.org, mhiramat@kernel.org, mingo@redhat.com, paulmck@kernel.org, peterz@infradead.org, rostedt@goodmis.org, senozhatsky@chromium.org, Alan Stern , Thomas Gleixner , vgoyal@redhat.com, vkuznets@redhat.com, Will Deacon , Alexander Gordeev , Andrea Parri , Ard Biesheuvel , Benjamin Herrenschmidt , Brian Norris , Christian Borntraeger , Christophe JAILLET , "David S. Miller" , Dexuan Cui , Doug Berger , Haiyang Zhang , Hari Bathini , Heiko Carstens , Justin Chen , "K. Y. Srinivasan" , Lee Jones , Markus Mayer , Michael Ellerman , Mihai Carabas , Nicholas Piggin , Paul Mackerras , Pavel Machek , Shile Zhang , Stephen Hemminger , Sven Schnelle , Thomas Bogendoerfer , Tianyu Lan , Vasily Gorbik , Wang ShaoBo , Wei Liu , zhenwei pi , Stephen Boyd Subject: Re: [PATCH 19/30] panic: Add the panic hypervisor notifier list Message-ID: References: <20220427224924.592546-1-gpiccoli@igalia.com> <20220427224924.592546-20-gpiccoli@igalia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-edac@vger.kernel.org On Tue 2022-05-17 13:37:58, Guilherme G. Piccoli wrote: > On 17/05/2022 10:28, Petr Mladek wrote: > > [...] > >>> Disagree here. I'm looping Google maintainers, so they can comment. > >>> (CCed Evan, David, Julius) > >>> > >>> This notifier is clearly a hypervisor notification mechanism. I've fixed > >>> a locking stuff there (in previous patch), I feel it's low-risk but even > >>> if it's mid-risk, the class of such callback remains a perfect fit with > >>> the hypervisor list IMHO. > >> > >> This logs a panic to our "eventlog", a tiny logging area in SPI flash > >> for critical and power-related events. In some cases this ends up > >> being the only clue we get in a Chromebook feedback report that a > >> panic occurred, so from my perspective moving it to the front of the > >> line seems like a good idea. > > > > IMHO, this would really better fit into the pre-reboot notifier list: > > > > + the callback stores the log so it is similar to kmsg_dump() > > or console_flush_on_panic() > > > > + the callback should be proceed after "info" notifiers > > that might add some other useful information. > > > > Honestly, I am not sure what exactly hypervisor callbacks do. But I > > think that they do not try to extract the kernel log because they > > would need to handle the internal format. > > > > I guess the main point in your response is : "I am not sure what exactly > hypervisor callbacks do". We need to be sure about the semantics of such > list, and agree on that. > > So, my opinion about this first list, that we call "hypervisor list", > is: it contains callbacks that > > (1) should run early, preferably before kdump (or even if kdump isn't > set, should run ASAP); > > (2) these callbacks perform some communication with an abstraction that > runs "below" the kernel, like a firmware or hypervisor. Classic example: > pvpanic, that communicates with VMM (usually qemu) and allow such VMM to > snapshot the full guest memory, for example. > > (3) Should be low-risk. What defines risk is the level of reliability of > subsequent operations - if the callback have 50% of chance of "bricking" > the system totally and prevent kdump / kmsg_dump() / reboot , this is > high risk one for example. > > Some good fits IMO: pvpanic, sstate_panic_event() [sparc], fadump in > powerpc, etc. > > So, this is a good case for the Google notifier as well - it's not > collecting data like the dmesg (hence your second bullet seems to not > apply here, info notifiers won't add info to be collected by gsmi). It > is a firmware/hypervisor/whatever-gsmi-is notification mechanism, that > tells such "lower" abstraction a panic occurred. It seems low risk and > we want it to run ASAP, if possible. " > >> This logs a panic to our "eventlog", a tiny logging area in SPI flash > >> for critical and power-related events. In some cases this ends up I see. I somehow assumed that it was about the kernel log because Evans wrote: "This logs a panic to our "eventlog", a tiny logging area in SPI flash for critical and power-related events. In some cases this ends up" Anyway, I would distinguish it the following way. + If the notifier is preserving kernel log then it should be ideally treated as kmsg_dump(). + It the notifier is saving another debugging data then it better fits into the "hypervisor" notifier list. Regarding the reliability. From my POV, any panic notifier enabled in a generic kernel should be reliable with more than 99,9%. Otherwise, they should not be in the notifier list at all. An exception would be a platform-specific notifier that is called only on some specific platform and developers maintaining this platform agree on this. The value "99,9%" is arbitrary. I am not sure if it is realistic even in the other code, for example, console_flush_on_panic() or emergency_restart(). I just want to point out that the border should be rather high. Otherwise we would back in the situation where people would want to disable particular notifiers. Best Regards, Petr From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 18 May 2022 09:33:57 +0200 From: Petr Mladek Subject: Re: [PATCH 19/30] panic: Add the panic hypervisor notifier list Message-ID: References: <20220427224924.592546-1-gpiccoli@igalia.com> <20220427224924.592546-20-gpiccoli@igalia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-um" Errors-To: linux-um-bounces+geert=linux-m68k.org@lists.infradead.org To: "Guilherme G. Piccoli" Cc: Evan Green , David Gow , Julius Werner , Scott Branden , bcm-kernel-feedback-list@broadcom.com, Sebastian Reichel , Linux PM , Florian Fainelli , Andrew Morton , bhe@redhat.com, kexec@lists.infradead.org, LKML , linuxppc-dev@lists.ozlabs.org, linux-alpha@vger.kernel.org, linux-arm Mailing List , linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-leds@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, linux-tegra@vger.kernel.org, linux-um@lists.infradead.org, linux-xtensa@linux-xtensa.org, netdev@vger.kernel.org, openipmi-developer@lists.sourceforge.net, rcu@vger.kernel.org, sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org, x86@kernel.org, kernel-dev@igalia.com, kernel@gpiccoli.net, halves@canonical.com, fabiomirmar@gmail.com, alejandro.j.jimenez@oracle.com, Andy Shevchenko , Arnd Bergmann , Borislav Petkov , Jonathan Corbet , d.hatayama@jp.fujitsu.com, dave.hansen@linux.intel.com, dyoung@redhat.com, feng.tang@intel.com, Greg Kroah-Hartman , mikelley@microsoft.com, hidehiro.kawai.ez@hitachi.com, jgross@suse.com, john.ogness@linutronix.de, Kees Cook , luto@kernel.org, mhiramat@kernel.org, mingo@redhat.com, paulmck@kernel.org, peterz@infradead.org, rostedt@goodmis.org, senozhatsky@chromium.org, Alan Stern , Thomas Gleixner , vgoyal@redhat.com, vkuznets@redhat.com, Will Deacon , Alexander Gordeev , Andrea Parri , Ard Biesheuvel , Benjamin Herrenschmidt , Brian Norris , Christian Borntraeger , Christophe JAILLET , "David S. Miller" , Dexuan Cui , Doug Berger , Haiyang Zhang , Hari Bathini , Heiko Carstens , Justin Chen , "K. Y. Srinivasan" , Lee Jones , Markus Mayer , Michael Ellerman , Mihai Carabas , Nicholas Piggin , Paul Mackerras , Pavel Machek , Shile Zhang , Stephen Hemminger , Sven Schnelle , Thomas Bogendoerfer , Tianyu Lan , Vasily Gorbik , Wang ShaoBo , Wei Liu , zhenwei pi , Stephen Boyd On Tue 2022-05-17 13:37:58, Guilherme G. Piccoli wrote: > On 17/05/2022 10:28, Petr Mladek wrote: > > [...] > >>> Disagree here. I'm looping Google maintainers, so they can comment. > >>> (CCed Evan, David, Julius) > >>> > >>> This notifier is clearly a hypervisor notification mechanism. I've fixed > >>> a locking stuff there (in previous patch), I feel it's low-risk but even > >>> if it's mid-risk, the class of such callback remains a perfect fit with > >>> the hypervisor list IMHO. > >> > >> This logs a panic to our "eventlog", a tiny logging area in SPI flash > >> for critical and power-related events. In some cases this ends up > >> being the only clue we get in a Chromebook feedback report that a > >> panic occurred, so from my perspective moving it to the front of the > >> line seems like a good idea. > > > > IMHO, this would really better fit into the pre-reboot notifier list: > > > > + the callback stores the log so it is similar to kmsg_dump() > > or console_flush_on_panic() > > > > + the callback should be proceed after "info" notifiers > > that might add some other useful information. > > > > Honestly, I am not sure what exactly hypervisor callbacks do. But I > > think that they do not try to extract the kernel log because they > > would need to handle the internal format. > > > > I guess the main point in your response is : "I am not sure what exactly > hypervisor callbacks do". We need to be sure about the semantics of such > list, and agree on that. > > So, my opinion about this first list, that we call "hypervisor list", > is: it contains callbacks that > > (1) should run early, preferably before kdump (or even if kdump isn't > set, should run ASAP); > > (2) these callbacks perform some communication with an abstraction that > runs "below" the kernel, like a firmware or hypervisor. Classic example: > pvpanic, that communicates with VMM (usually qemu) and allow such VMM to > snapshot the full guest memory, for example. > > (3) Should be low-risk. What defines risk is the level of reliability of > subsequent operations - if the callback have 50% of chance of "bricking" > the system totally and prevent kdump / kmsg_dump() / reboot , this is > high risk one for example. > > Some good fits IMO: pvpanic, sstate_panic_event() [sparc], fadump in > powerpc, etc. > > So, this is a good case for the Google notifier as well - it's not > collecting data like the dmesg (hence your second bullet seems to not > apply here, info notifiers won't add info to be collected by gsmi). It > is a firmware/hypervisor/whatever-gsmi-is notification mechanism, that > tells such "lower" abstraction a panic occurred. It seems low risk and > we want it to run ASAP, if possible. " > >> This logs a panic to our "eventlog", a tiny logging area in SPI flash > >> for critical and power-related events. In some cases this ends up I see. I somehow assumed that it was about the kernel log because Evans wrote: "This logs a panic to our "eventlog", a tiny logging area in SPI flash for critical and power-related events. In some cases this ends up" Anyway, I would distinguish it the following way. + If the notifier is preserving kernel log then it should be ideally treated as kmsg_dump(). + It the notifier is saving another debugging data then it better fits into the "hypervisor" notifier list. Regarding the reliability. From my POV, any panic notifier enabled in a generic kernel should be reliable with more than 99,9%. Otherwise, they should not be in the notifier list at all. An exception would be a platform-specific notifier that is called only on some specific platform and developers maintaining this platform agree on this. The value "99,9%" is arbitrary. I am not sure if it is realistic even in the other code, for example, console_flush_on_panic() or emergency_restart(). I just want to point out that the border should be rather high. Otherwise we would back in the situation where people would want to disable particular notifiers. Best Regards, Petr _______________________________________________ linux-um mailing list linux-um@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-um From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D439EC433EF for ; Wed, 18 May 2022 11:06:56 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4L39DG4Y8rz3byH for ; Wed, 18 May 2022 21:06:54 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=suse.com header.i=@suse.com header.a=rsa-sha256 header.s=susede1 header.b=apiSUoAw; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=suse.com (client-ip=195.135.220.29; helo=smtp-out2.suse.de; envelope-from=pmladek@suse.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.com header.i=@suse.com header.a=rsa-sha256 header.s=susede1 header.b=apiSUoAw; dkim-atps=neutral Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4L34Vn1S8Zz3bZP for ; Wed, 18 May 2022 17:34:07 +1000 (AEST) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 8990E1F9A4; Wed, 18 May 2022 07:34:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1652859243; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YR5rOhmSURb4d/IKXy5VZg8z1ImD01Us4xBpVFtyoqU=; b=apiSUoAwUbqeAccPNQ19K+RPeAfjpiC+Kw09kP9gYD08Bjqn9sy6l8W4iwyq6IQeI6Aplp +YIfJeRJqbcdQptzveOcUb8Q0R7nuJcbMiyUrGxQWxZdDf/W1Ubmr612J8Zmj7lPwrkx5l wG1cTFsW35nAwzkY9hs+1GgrMVmINpw= Received: from suse.cz (unknown [10.100.201.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id AA6042C142; Wed, 18 May 2022 07:33:59 +0000 (UTC) Date: Wed, 18 May 2022 09:33:57 +0200 From: Petr Mladek To: "Guilherme G. Piccoli" Subject: Re: [PATCH 19/30] panic: Add the panic hypervisor notifier list Message-ID: References: <20220427224924.592546-1-gpiccoli@igalia.com> <20220427224924.592546-20-gpiccoli@igalia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Mailman-Approved-At: Wed, 18 May 2022 21:06:24 +1000 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paul Mackerras , Justin Chen , Pavel Machek , Alexander Gordeev , "K. Y. Srinivasan" , Wei Liu , Alan Stern , xen-devel@lists.xenproject.org, Christian Borntraeger , Linux PM , linux-um@lists.infradead.org, Nicholas Piggin , Stephen Boyd , luto@kernel.org, Mihai Carabas , Thomas Gleixner , Greg Kroah-Hartman , LKML , senozhatsky@chromium.org, d.hatayama@jp.fujitsu.com, mhiramat@kernel.org, Andrew Morton , linux-hyperv@vger.kernel.org, dave.hansen@linux.intel.com, linux-s390@vger.kernel.org, Stephen Hemminger , Vasily Gorbik , vgoyal@redhat.com, Sven Schnelle , Andrea Parri , linux-xtensa@linux-xtensa.org, john.ogness@linutronix.de, Scott Branden , Doug Berger , Markus Mayer , hidehiro.kawai.ez@hitachi.com, linux-arm Mailing List , kernel-dev@igalia.com, fabiomirmar@gmail.com, halves@canonical.com, alejandro.j.jimenez@oracle.com, feng.tang@intel.com, zhenwei pi , Will Deacon , Florian Fainelli , bhe@redhat.com, Jonathan Corbet , Dexuan Cui , Evan Green , bcm-kernel-feedback-list@broadcom.com, Tianyu Lan , Kees Cook , Arnd Bergmann , Haiyang Zhang , rostedt@goodmis.org, rcu@vger.kernel.org, Borislav Petkov , openipmi-developer@lists.sourceforge.net, Thomas Bogendoerfer , linux-parisc@vger.kernel.org, Sebastian Reichel , linux-alpha@vger.kernel.org, Brian Norris , "David S. Miller" , peterz@infradead.org, linux-remoteproc@vger.kernel.org, mikelley@microsoft.com, sparclinux@vger.kernel.org, Lee Jones , Ard Biesheuvel , linux-leds@vger.kernel.org, x86@kernel.org, mingo@redhat.com, dyoung@redhat.com, paulmck@kernel.org, Heiko Carstens , Shile Zhang , Wang ShaoBo , Christophe JAILLET , David Gow , linux-tegra@vger.kernel.org, Andy Shevchenko , Hari Bathini , linux-edac@vger.kernel.org, jgross@suse.com, netdev@vger.kernel.org, kernel@gpiccoli.net, kexec@lists.infradead.org, linux-mips@vger.kernel.org, Julius Werner , vkuznets@redhat.com, linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Tue 2022-05-17 13:37:58, Guilherme G. Piccoli wrote: > On 17/05/2022 10:28, Petr Mladek wrote: > > [...] > >>> Disagree here. I'm looping Google maintainers, so they can comment. > >>> (CCed Evan, David, Julius) > >>> > >>> This notifier is clearly a hypervisor notification mechanism. I've fixed > >>> a locking stuff there (in previous patch), I feel it's low-risk but even > >>> if it's mid-risk, the class of such callback remains a perfect fit with > >>> the hypervisor list IMHO. > >> > >> This logs a panic to our "eventlog", a tiny logging area in SPI flash > >> for critical and power-related events. In some cases this ends up > >> being the only clue we get in a Chromebook feedback report that a > >> panic occurred, so from my perspective moving it to the front of the > >> line seems like a good idea. > > > > IMHO, this would really better fit into the pre-reboot notifier list: > > > > + the callback stores the log so it is similar to kmsg_dump() > > or console_flush_on_panic() > > > > + the callback should be proceed after "info" notifiers > > that might add some other useful information. > > > > Honestly, I am not sure what exactly hypervisor callbacks do. But I > > think that they do not try to extract the kernel log because they > > would need to handle the internal format. > > > > I guess the main point in your response is : "I am not sure what exactly > hypervisor callbacks do". We need to be sure about the semantics of such > list, and agree on that. > > So, my opinion about this first list, that we call "hypervisor list", > is: it contains callbacks that > > (1) should run early, preferably before kdump (or even if kdump isn't > set, should run ASAP); > > (2) these callbacks perform some communication with an abstraction that > runs "below" the kernel, like a firmware or hypervisor. Classic example: > pvpanic, that communicates with VMM (usually qemu) and allow such VMM to > snapshot the full guest memory, for example. > > (3) Should be low-risk. What defines risk is the level of reliability of > subsequent operations - if the callback have 50% of chance of "bricking" > the system totally and prevent kdump / kmsg_dump() / reboot , this is > high risk one for example. > > Some good fits IMO: pvpanic, sstate_panic_event() [sparc], fadump in > powerpc, etc. > > So, this is a good case for the Google notifier as well - it's not > collecting data like the dmesg (hence your second bullet seems to not > apply here, info notifiers won't add info to be collected by gsmi). It > is a firmware/hypervisor/whatever-gsmi-is notification mechanism, that > tells such "lower" abstraction a panic occurred. It seems low risk and > we want it to run ASAP, if possible. " > >> This logs a panic to our "eventlog", a tiny logging area in SPI flash > >> for critical and power-related events. In some cases this ends up I see. I somehow assumed that it was about the kernel log because Evans wrote: "This logs a panic to our "eventlog", a tiny logging area in SPI flash for critical and power-related events. In some cases this ends up" Anyway, I would distinguish it the following way. + If the notifier is preserving kernel log then it should be ideally treated as kmsg_dump(). + It the notifier is saving another debugging data then it better fits into the "hypervisor" notifier list. Regarding the reliability. From my POV, any panic notifier enabled in a generic kernel should be reliable with more than 99,9%. Otherwise, they should not be in the notifier list at all. An exception would be a platform-specific notifier that is called only on some specific platform and developers maintaining this platform agree on this. The value "99,9%" is arbitrary. I am not sure if it is realistic even in the other code, for example, console_flush_on_panic() or emergency_restart(). I just want to point out that the border should be rather high. Otherwise we would back in the situation where people would want to disable particular notifiers. Best Regards, Petr