linux-alpha.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Marc Zyngier <maz@kernel.org>
To: "Guilherme G. Piccoli" <gpiccoli@igalia.com>
Cc: akpm@linux-foundation.org, bhe@redhat.com, pmladek@suse.com,
	kexec@lists.infradead.org, linux-kernel@vger.kernel.org,
	bcm-kernel-feedback-list@broadcom.com,
	coresight@lists.linaro.org, linuxppc-dev@lists.ozlabs.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, linux-edac@vger.kernel.org,
	linux-hyperv@vger.kernel.org, linux-leds@vger.kernel.org,
	linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org,
	linux-pm@vger.kernel.org, linux-remoteproc@vger.kernel.org,
	linux-s390@vger.kernel.org, linux-tegra@vger.kernel.org,
	linux-um@lists.infradead.org, linux-xtensa@linux-xtensa.org,
	netdev@vger.kernel.org, openipmi-developer@lists.sourceforge.net,
	rcu@vger.kernel.org, sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org, x86@kernel.org,
	kernel-dev@igalia.com, k
Subject: Re: [PATCH 02/30] ARM: kexec: Disable IRQs/FIQs also on crash CPUs shutdown path
Date: Fri, 29 Apr 2022 19:20:29 +0100	[thread overview]
Message-ID: <87mtg392fm.wl-maz@kernel.org> (raw)
In-Reply-To: <20220427224924.592546-3-gpiccoli@igalia.com>

On Wed, 27 Apr 2022 23:48:56 +0100,
"Guilherme G. Piccoli" <gpiccoli@igalia.com> wrote:
> 
> Currently the regular CPU shutdown path for ARM disables IRQs/FIQs
> in the secondary CPUs - smp_send_stop() calls ipi_cpu_stop(), which
> is responsible for that. This makes sense, since we're turning off
> such CPUs, putting them in an endless busy-wait loop.
> 
> Problem is that there is an alternative path for disabling CPUs,
> in the form of function crash_smp_send_stop(), used for kexec/panic
> paths. This functions relies in a SMP call that also triggers a
> busy-wait loop [at machine_crash_nonpanic_core()], but *without*
> disabling interrupts. This might lead to odd scenarios, like early
> interrupts in the boot of kexec'd kernel or even interrupts in
> other CPUs while the main one still works in the panic path and
> assumes all secondary CPUs are (really!) off.
> 
> This patch mimics the ipi_cpu_stop() interrupt disable mechanism
> in the crash CPU shutdown path, hence disabling IRQs/FIQs in all
> secondary CPUs in the kexec/panic path as well.
> 
> Cc: Marc Zyngier <maz@kernel.org>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
> ---
>  arch/arm/kernel/machine_kexec.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/arch/arm/kernel/machine_kexec.c b/arch/arm/kernel/machine_kexec.c
> index f567032a09c0..ef788ee00519 100644
> --- a/arch/arm/kernel/machine_kexec.c
> +++ b/arch/arm/kernel/machine_kexec.c
> @@ -86,6 +86,9 @@ void machine_crash_nonpanic_core(void *unused)
>  	set_cpu_online(smp_processor_id(), false);
>  	atomic_dec(&waiting_for_crash_ipi);
>  
> +	local_fiq_disable();
> +	local_irq_disable();
> +

My expectations would be that, since we're getting here using an IPI,
interrupts are already masked. So what reenabled them the first place?

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

  parent reply	other threads:[~2022-04-29 18:20 UTC|newest]

Thread overview: 179+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-04-27 22:48 [PATCH 00/30] The panic notifiers refactor Guilherme G. Piccoli
2022-04-27 22:48 ` [PATCH 01/30] x86/crash, reboot: Avoid re-disabling VMX in all CPUs on crash/restart Guilherme G. Piccoli
2022-05-09 12:32   ` [PATCH 01/30] x86/crash,reboot: " Guilherme G. Piccoli
     [not found]   ` <20220427224924.592546-2-gpiccoli-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-09 15:52     ` Sean Christopherson
2022-05-10 20:11       ` Guilherme G. Piccoli
2022-04-27 22:48 ` [PATCH 02/30] ARM: kexec: Disable IRQs/FIQs also on crash CPUs shutdown path Guilherme G. Piccoli
2022-04-29 16:26   ` Michael Kelley (LINUX)
2022-04-29 18:20   ` Marc Zyngier [this message]
     [not found]     ` <87mtg392fm.wl-maz-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2022-04-29 21:38       ` Guilherme G. Piccoli
2022-04-29 21:45         ` Russell King (Oracle)
2022-04-29 21:56           ` Guilherme G. Piccoli
2022-04-29 22:00           ` Marc Zyngier
2022-04-27 22:48 ` [PATCH 03/30] notifier: Add panic notifiers info and purge trailing whitespaces Guilherme G. Piccoli
2022-04-27 22:48 ` [PATCH 04/30] firmware: google: Convert regular spinlock into trylock on panic path Guilherme G. Piccoli
2022-05-03 18:03   ` Evan Green
     [not found]     ` <CAE=gft5Pq25L4KFoPWbftkPF-JN1ex2yws77mMJ4GQnn9W0L2g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2022-05-03 19:12       ` Guilherme G. Piccoli
     [not found]         ` <adcf6d0e-c37c-6ede-479e-29959d03d8c0-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-03 21:56           ` Evan Green
     [not found]             ` <CAE=gft623NxqetRssrZnaRmJLSP4BT5=-sVVwtYoHuspO_gULQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2022-05-04 12:45               ` Guilherme G. Piccoli
2022-05-10 11:38           ` Petr Mladek
2022-05-10 13:04             ` Guilherme G. Piccoli
2022-05-10 17:20             ` Steven Rostedt
2022-05-10 19:40               ` John Ogness
2022-05-11 11:13                 ` Petr Mladek
2022-04-27 22:48 ` [PATCH 05/30] misc/pvpanic: " Guilherme G. Piccoli
2022-05-10 12:14   ` Petr Mladek
2022-05-10 13:00     ` Guilherme G. Piccoli
2022-05-17 10:58       ` Petr Mladek
2022-05-17 13:03         ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 06/30] soc: bcm: brcmstb: Document panic notifier action and remove useless header Guilherme G. Piccoli
     [not found]   ` <20220427224924.592546-7-gpiccoli-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-02 15:38     ` Florian Fainelli
     [not found]       ` <a02821ab-db4f-5bff-2a98-7d74032a0652-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2022-05-02 15:47         ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 07/30] mips: ip22: Reword PANICED to PANICKED " Guilherme G. Piccoli
2022-05-04 20:32   ` Thomas Bogendoerfer
2022-05-04 21:26     ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 08/30] powerpc/setup: Refactor/untangle panic notifiers Guilherme G. Piccoli
2022-05-05 18:55   ` Hari Bathini
2022-05-05 19:28     ` Guilherme G. Piccoli
2022-05-09 12:50     ` Guilherme G. Piccoli
2022-05-10 13:53       ` Michael Ellerman
2022-05-10 14:10         ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 09/30] coresight: cpu-debug: Replace mutex with mutex_trylock on panic notifier Guilherme G. Piccoli
     [not found]   ` <20220427224924.592546-10-gpiccoli-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-04-28  8:11     ` Suzuki K Poulose
2022-04-29 14:01       ` Guilherme G. Piccoli
     [not found]       ` <3cafe4fd-8a0b-2633-44a3-2995abd6c38c-5wv7dgnIgG8@public.gmane.org>
2022-05-09 13:09         ` Guilherme G. Piccoli
     [not found]           ` <65f24bc5-2211-0139-ee12-b2608e81ceb1-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-09 16:14             ` Suzuki K Poulose
     [not found]               ` <d9ec6f31-6125-0723-b7d7-5898abeb3289-5wv7dgnIgG8@public.gmane.org>
2022-05-09 16:26                 ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 10/30] alpha: Clean-up the panic notifier code Guilherme G. Piccoli
     [not found]   ` <20220427224924.592546-11-gpiccoli-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-09 14:13     ` Guilherme G. Piccoli
2022-05-10 14:16       ` Petr Mladek
2022-05-11 20:10         ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 11/30] um: Improve panic notifiers consistency and ordering Guilherme G. Piccoli
2022-05-10 14:28   ` Petr Mladek
2022-05-11 20:22     ` Guilherme G. Piccoli
2022-05-13 14:44       ` Johannes Berg
     [not found]         ` <1760d499824f9ef053af7a8dac04b48ab7d7fd3d.camel-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
2022-05-15 22:12           ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 12/30] parisc: Replace regular spinlock with spin_trylock on panic path Guilherme G. Piccoli
2022-04-28 16:55   ` Helge Deller
2022-04-29 14:34     ` Guilherme G. Piccoli
2022-05-23 20:40     ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 13/30] s390/consoles: Improve panic notifiers reliability Guilherme G. Piccoli
2022-04-29 18:46   ` Heiko Carstens
2022-04-29 19:31     ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 14/30] panic: Properly identify the panic event to the notifiers' callbacks Guilherme G. Piccoli
2022-05-10 15:16   ` Petr Mladek via Openipmi-developer
2022-05-10 16:16     ` Guilherme G. Piccoli
2022-05-17 13:11       ` Petr Mladek
2022-05-17 15:19         ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 15/30] bus: brcmstb_gisb: Clean-up panic/die notifiers Guilherme G. Piccoli
2022-05-02 15:38   ` Florian Fainelli
2022-05-02 15:50     ` Guilherme G. Piccoli
2022-05-10 15:28   ` Petr Mladek
2022-05-17 15:32     ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 16/30] drivers/hv/vmbus, video/hyperv_fb: Untangle and refactor Hyper-V panic notifiers Guilherme G. Piccoli
2022-04-29 17:16   ` Michael Kelley (LINUX)
2022-04-29 22:35     ` Guilherme G. Piccoli
2022-05-03 18:13       ` Michael Kelley (LINUX)
2022-05-03 18:57         ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 17/30] tracing: Improve panic/die notifiers Guilherme G. Piccoli
2022-04-29  9:22   ` Sergei Shtylyov
2022-04-29 13:23     ` Steven Rostedt
2022-04-29 13:46       ` Guilherme G. Piccoli
2022-04-29 13:56         ` Steven Rostedt
2022-04-29 14:44           ` Guilherme G. Piccoli
     [not found]   ` <20220427224924.592546-18-gpiccoli-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-11 11:45     ` Petr Mladek
2022-05-17 15:33       ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 18/30] notifier: Show function names on notifier routines if DEBUG_NOTIFIERS is set Guilherme G. Piccoli
2022-04-28  1:01   ` Xiaoming Ni
     [not found]     ` <9f44aae6-ec00-7ede-ec19-6e67ceb74510-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2022-04-29 19:38       ` Guilherme G. Piccoli
2022-05-10 17:29     ` Steven Rostedt
2022-05-16 16:14       ` Guilherme G. Piccoli
2022-04-29 16:27   ` Michael Kelley (LINUX)
2022-04-27 22:49 ` [PATCH 19/30] panic: Add the panic hypervisor notifier list Guilherme G. Piccoli
2022-04-29 17:30   ` Michael Kelley (LINUX)
2022-04-29 18:04     ` Guilherme G. Piccoli
     [not found]       ` <0147d038-571b-0802-c210-ccd4d52cd5dd-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-03 17:44         ` Michael Kelley (LINUX)
2022-05-03 17:56           ` Guilherme G. Piccoli
     [not found]   ` <20220427224924.592546-20-gpiccoli-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-16 14:01     ` Petr Mladek
2022-05-16 15:06       ` Guilherme G. Piccoli
2022-05-16 16:02         ` Evan Green
     [not found]           ` <CAE=gft7ds+dHfEkRz8rnSH1EbTpGTpKbi5Wxj9DW0Jr5mX_j4w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2022-05-17 13:28             ` Petr Mladek
2022-05-17 16:37               ` Guilherme G. Piccoli
     [not found]                 ` <b9ec2fc8-216f-f261-8417-77b6dd95e25c-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-18  7:33                   ` Petr Mladek
2022-05-18 13:24                     ` Guilherme G. Piccoli
2022-05-17 13:57         ` Petr Mladek
2022-05-17 16:42           ` Guilherme G. Piccoli
2022-05-18  7:38             ` Petr Mladek
2022-05-18 13:09               ` Guilherme G. Piccoli
     [not found]             ` <d72b9aab-675c-ac89-b73a-b1de4a0b722d-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-18 22:17               ` Scott Branden
2022-05-19 12:19                 ` Guilherme G. Piccoli
2022-05-19 19:20                   ` Scott Branden
     [not found]                     ` <d1cc0bee-2a98-0c2e-8796-6fb7fae6b803-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
2022-05-23 14:56                       ` Guilherme G. Piccoli
     [not found]                         ` <0fac8c71-6f18-d15c-23f5-075dbc45f3f9-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-24  8:04                           ` Petr Mladek
2022-05-18  7:58           ` Petr Mladek
2022-05-18 13:16             ` Guilherme G. Piccoli
2022-05-19  7:03               ` Petr Mladek
2022-05-19 12:07                 ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 20/30] panic: Add the panic informational " Guilherme G. Piccoli
2022-04-27 23:49   ` Paul E. McKenney
2022-04-28  8:14   ` Suzuki K Poulose
     [not found]     ` <7956ab00-66b6-bd89-dcc0-f10cf2741e4d-5wv7dgnIgG8@public.gmane.org>
2022-04-29 14:50       ` Guilherme G. Piccoli
2022-05-16 14:11   ` Petr Mladek
2022-05-16 14:28     ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 21/30] panic: Introduce the panic pre-reboot " Guilherme G. Piccoli
2022-04-28 14:13   ` Alex Elder
2022-04-29 16:04   ` Max Filippov
2022-04-29 19:34     ` Guilherme G. Piccoli
     [not found]   ` <20220427224924.592546-22-gpiccoli-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-04-28 16:26     ` Corey Minyard
     [not found]       ` <20220428162616.GE442787-cSpEYAXvYQ3R7s880joybQ@public.gmane.org>
2022-04-29 15:18         ` Guilherme G. Piccoli
2022-05-16 14:33     ` Petr Mladek
2022-05-16 16:05       ` Guilherme G. Piccoli
2022-05-16 16:18         ` Luck, Tony
     [not found]           ` <bed66b9467254a5a8bafc1983dad643a-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2022-05-16 16:33             ` Guilherme G. Piccoli
2022-05-17 14:11               ` Petr Mladek via Openipmi-developer
2022-05-17 16:45                 ` Guilherme G. Piccoli
2022-05-17 17:02                   ` Luck, Tony
2022-05-17 18:12                     ` Guilherme G. Piccoli
2022-05-17 19:07                       ` Luck, Tony
2022-04-27 22:49 ` [PATCH 22/30] panic: Introduce the panic post-reboot " Guilherme G. Piccoli
2022-05-09 14:16   ` Guilherme G. Piccoli
2022-05-11 16:45     ` Heiko Carstens
2022-05-11 19:58       ` Guilherme G. Piccoli
2022-05-16 14:45   ` Petr Mladek
2022-05-16 16:08     ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 23/30] printk: kmsg_dump: Introduce helper to inform number of dumpers Guilherme G. Piccoli
2022-05-10 17:40   ` Steven Rostedt
     [not found]     ` <20220510134014.3923ccba-f9ZlEuEWxVcJvu8Pb33WZ0EMvNT87kid@public.gmane.org>
2022-05-11 20:03       ` Guilherme G. Piccoli
     [not found]         ` <c8818906-f113-82b6-b58b-d47ae0c16b4f-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-16 14:50           ` Petr Mladek
2022-05-16 16:09             ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 24/30] panic: Refactor the panic path Guilherme G. Piccoli
2022-04-28  0:28   ` Randy Dunlap
     [not found]     ` <4fe85e9c-4e96-e9d5-9fd8-f062bafcda4f-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2022-04-29 16:04       ` Guilherme G. Piccoli
     [not found]         ` <7518924e-5bb4-e6e9-0e3e-3f5cb03bf946-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-09 14:25           ` Guilherme G. Piccoli
2022-04-29 17:53   ` Michael Kelley (LINUX)
2022-04-29 20:38     ` Guilherme G. Piccoli
     [not found]       ` <50178dfb-8e94-f35f-09c3-22fe197550ef-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-03 17:31         ` Michael Kelley (LINUX)
2022-05-03 18:06           ` Guilherme G. Piccoli
2022-05-09 15:16   ` d.hatayama
     [not found]     ` <TYAPR01MB6507D01F5694BC33628BB7DB95C69-0qPFe2CvyVTfsDfOwIA1BXcolHNk5qUtvxpqHgZTriW3zl9H0oFU5g@public.gmane.org>
2022-05-09 16:39       ` Guilherme G. Piccoli
2022-05-12 14:03   ` Petr Mladek
2022-05-15 22:47     ` Guilherme G. Piccoli
     [not found]       ` <d313eec2-96b6-04e3-35cd-981f103d010e-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-05-16 10:21         ` Petr Mladek
2022-05-16 16:32           ` Guilherme G. Piccoli
2022-05-19 23:45       ` Baoquan He
2022-05-20 11:23         ` Guilherme G. Piccoli
2022-05-24  8:01           ` Petr Mladek
2022-05-24 10:18             ` Baoquan He
2022-05-24  8:32           ` Baoquan He
2022-05-24 14:44   ` Eric W. Biederman
2022-05-26 16:25     ` Guilherme G. Piccoli
     [not found]       ` <0d084eed-4781-c815-29c7-ac62c498e216-wEGTBA9jqPzQT0dZR+AlfA@public.gmane.org>
2022-06-14 14:36         ` Petr Mladek
2022-06-15  9:36           ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 25/30] panic, printk: Add console flush parameter and convert panic_print to a notifier Guilherme G. Piccoli
2022-05-16 14:56   ` Petr Mladek
2022-05-16 16:11     ` Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 26/30] Drivers: hv: Do not force all panic notifiers to execute before kdump Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 27/30] powerpc: " Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 28/30] panic: Unexport crash_kexec_post_notifiers Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 29/30] powerpc: ps3, pseries: Avoid duplicate call to kmsg_dump() on panic Guilherme G. Piccoli
2022-04-27 22:49 ` [PATCH 30/30] um: Avoid duplicate call to kmsg_dump() Guilherme G. Piccoli

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87mtg392fm.wl-maz@kernel.org \
    --to=maz@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=bcm-kernel-feedback-list@broadcom.com \
    --cc=bhe@redhat.com \
    --cc=coresight@lists.linaro.org \
    --cc=gpiccoli@igalia.com \
    --cc=kernel-dev@igalia.com \
    --cc=kexec@lists.infradead.org \
    --cc=linux-alpha@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-edac@vger.kernel.org \
    --cc=linux-hyperv@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-leds@vger.kernel.org \
    --cc=linux-mips@vger.kernel.org \
    --cc=linux-parisc@vger.kernel.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=linux-remoteproc@vger.kernel.org \
    --cc=linux-s390@vger.kernel.org \
    --cc=linux-tegra@vger.kernel.org \
    --cc=linux-um@lists.infradead.org \
    --cc=linux-xtensa@linux-xtensa.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=netdev@vger.kernel.org \
    --cc=openipmi-developer@lists.sourceforge.net \
    --cc=pmladek@suse.com \
    --cc=rcu@vger.kernel.org \
    --cc=sparclinux@vger.kernel.org \
    --cc=x86@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).