From: Ingo Molnar <mingo@kernel.org>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Chris Metcalf <cmetcalf@ezchip.com>,
Steven Rostedt <rostedt@goodmis.org>,
Gilad Ben Yossef <giladb@ezchip.com>,
Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Rik van Riel <riel@redhat.com>, Tejun Heo <tj@kernel.org>,
Frederic Weisbecker <fweisbec@gmail.com>,
Thomas Gleixner <tglx@linutronix.de>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Christoph Lameter <cl@linux.com>,
"Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>,
linux-doc@vger.kernel.org, linux-api@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 0/6] support "dataplane" mode for nohz_full
Date: Sat, 9 May 2015 09:05:38 +0200 [thread overview]
Message-ID: <20150509070538.GA9413@gmail.com> (raw)
In-Reply-To: <20150508161909.308d60e21f6b83b897174276@linux-foundation.org>
* Andrew Morton <akpm@linux-foundation.org> wrote:
> On Fri, 8 May 2015 19:11:10 -0400 Chris Metcalf <cmetcalf@ezchip.com> wrote:
>
> > On 5/8/2015 5:22 PM, Steven Rostedt wrote:
> > > On Fri, 8 May 2015 14:18:24 -0700
> > > Andrew Morton <akpm@linux-foundation.org> wrote:
> > >
> > >> On Fri, 8 May 2015 13:58:41 -0400 Chris Metcalf <cmetcalf@ezchip.com> wrote:
> > >>
> > >>> A prctl() option (PR_SET_DATAPLANE) is added
> > >> Dumb question: what does the term "dataplane" mean in this context? I
> > >> can't see the relationship between those words and what this patch
> > >> does.
> > > I was thinking the same thing. I haven't gotten around to searching
> > > DATAPLANE yet.
> > >
> > > I would assume we want a name that is more meaningful for what is
> > > happening.
> >
> > The text in the commit message and the 0/6 cover letter do try to explain
> > the concept. The terminology comes, I think, from networking line cards,
> > where the "dataplane" is the part of the application that handles all the
> > fast path processing of network packets, and the "control plane" is the part
> > that handles routing updates, etc., generally slow-path stuff. I've probably
> > just been using the terms so long they seem normal to me.
> >
> > That said, what would be clearer? NO_HZ_STRICT as a superset of
> > NO_HZ_FULL? Or move away from the NO_HZ terminology a bit; after all,
> > we're talking about no interrupts of any kind, and maybe NO_HZ is too
> > limited in scope? So, NO_INTERRUPTS? USERSPACE_ONLY? Or look
> > to vendors who ship bare-metal runtimes and call it BARE_METAL?
> > Borrow the Tilera marketing name and call it ZERO_OVERHEAD?
> >
> > Maybe BARE_METAL seems most plausible -- after DATAPLANE, to me,
> > of course :-)
'baremetal' has uses in virtualization speak, so I think that would be
confusing.
> I like NO_INTERRUPTS. Simple, direct.
NO_HZ_PURE?
That's what it's really about: user-space wants to run exclusively, in
pure user-mode, without any interrupts.
So I don't like 'NO_HZ_NO_INTERRUPTS' for a couple of reasons:
- It is similar to a term we use in perf: PERF_PMU_CAP_NO_INTERRUPT.
- Another reason is that 'NO_INTERRUPTS', in most existing uses in
the kernel generally relates to some sort of hardware weakness,
limitation, a negative property: that we try to limp along without
having a hardware interrupt and have to poll. In other driver code
that uses variants of NO_INTERRUPT it appears to be similar. So I
think there's some confusion potential here.
- Here the fact that we don't disturb user-space is an absolutely
positive property, not a limitation, a kernel feature we work hard
to achieve. NO_HZ_PURE would convey that while NO_HZ_NO_INTERRUPTS
wouldn't.
- NO_HZ_NO_INTERRUPTS has a double negation, and it's also too long,
compared to NO_HZ_FULL or NO_HZ_PURE ;-) The term 'no HZ' already
expresses that we don't have periodic interruptions. We just
duplicate that information with NO_HZ_NO_INTERRUPTS, while
NO_HZ_FULL or NO_HZ_PURE qualifies it, makes it a stronger
property - which is what we want I think.
So I think we should either rename NO_HZ_FULL to NO_HZ_PURE, or keep
it at NO_HZ_FULL: because the intention of NO_HZ_FULL was always to be
such a 'zero overhead' mode of operation, where if user-space runs, it
won't get interrupted in any way.
There's no need to add yet another Kconfig variant - lets just enhance
the current stuff and maybe rename it to NO_HZ_PURE to better express
its intent.
Thanks,
Ingo
next prev parent reply other threads:[~2015-05-09 7:05 UTC|newest]
Thread overview: 236+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-08 17:58 [PATCH 0/6] support "dataplane" mode for nohz_full Chris Metcalf
2015-05-08 17:58 ` [PATCH 1/6] nohz_full: add support for "dataplane" mode Chris Metcalf
2015-05-08 17:58 ` [PATCH 2/6] nohz: dataplane: allow tick to be fully disabled for dataplane Chris Metcalf
2015-05-12 9:26 ` Peter Zijlstra
2015-05-12 13:12 ` Paul E. McKenney
2015-05-14 20:55 ` Chris Metcalf
2015-05-08 17:58 ` [PATCH 3/6] dataplane nohz: run softirqs synchronously on user entry Chris Metcalf
2015-05-09 7:04 ` Mike Galbraith
2015-05-11 20:13 ` Chris Metcalf
2015-05-12 2:21 ` Mike Galbraith
2015-05-12 9:28 ` Peter Zijlstra
2015-05-12 9:32 ` Peter Zijlstra
2015-05-12 13:08 ` Paul E. McKenney
2015-05-08 17:58 ` [PATCH 4/6] nohz: support PR_DATAPLANE_QUIESCE Chris Metcalf
2015-05-12 9:33 ` Peter Zijlstra
2015-05-12 9:50 ` Ingo Molnar
2015-05-12 10:38 ` Peter Zijlstra
2015-05-12 12:52 ` Ingo Molnar
2015-05-13 4:35 ` Andy Lutomirski
2015-05-13 17:51 ` Paul E. McKenney
2015-05-14 20:55 ` Chris Metcalf
2015-05-14 20:54 ` Chris Metcalf
2015-05-08 17:58 ` [PATCH 5/6] nohz: support PR_DATAPLANE_STRICT mode Chris Metcalf
2015-05-09 7:28 ` Andy Lutomirski
2015-05-09 10:37 ` Gilad Ben Yossef
2015-05-11 19:13 ` Chris Metcalf
2015-05-11 22:28 ` Andy Lutomirski
2015-05-12 21:06 ` Chris Metcalf
2015-05-12 22:23 ` Andy Lutomirski
2015-05-15 21:25 ` Chris Metcalf
2015-05-12 9:38 ` Peter Zijlstra
2015-05-12 13:20 ` Paul E. McKenney
2015-05-08 17:58 ` [PATCH 6/6] nohz: add dataplane_debug boot flag Chris Metcalf
2015-05-08 21:18 ` [PATCH 0/6] support "dataplane" mode for nohz_full Andrew Morton
2015-05-08 21:22 ` Steven Rostedt
2015-05-08 23:11 ` Chris Metcalf
2015-05-08 23:19 ` Andrew Morton
2015-05-09 7:05 ` Ingo Molnar [this message]
2015-05-09 7:19 ` Andy Lutomirski
2015-05-11 19:54 ` Chris Metcalf
2015-05-11 22:15 ` Andy Lutomirski
[not found] ` <55510885.9070101@ezchip.com>
2015-05-12 13:18 ` Paul E. McKenney
2015-05-09 7:19 ` Mike Galbraith
2015-05-09 10:18 ` Gilad Ben Yossef
2015-05-11 12:57 ` Steven Rostedt
2015-05-11 15:36 ` Frederic Weisbecker
2015-05-11 19:19 ` Mike Galbraith
2015-05-11 19:25 ` Chris Metcalf
2015-05-12 1:47 ` Mike Galbraith
2015-05-12 4:35 ` Mike Galbraith
2015-05-15 15:05 ` Chris Metcalf
2015-05-15 18:44 ` Mike Galbraith
2015-05-26 19:51 ` Chris Metcalf
2015-05-27 3:28 ` Mike Galbraith
2015-05-11 17:19 ` Paul E. McKenney
2015-05-11 17:27 ` Andrew Morton
2015-05-11 17:33 ` Frederic Weisbecker
2015-05-11 18:00 ` Steven Rostedt
2015-05-11 18:09 ` Chris Metcalf
2015-05-11 18:36 ` Steven Rostedt
2015-05-12 9:10 ` CONFIG_ISOLATION=y (was: [PATCH 0/6] support "dataplane" mode for nohz_full) Ingo Molnar
2015-05-12 11:48 ` Peter Zijlstra
2015-05-12 12:34 ` Ingo Molnar
2015-05-12 12:39 ` Peter Zijlstra
2015-05-12 12:43 ` Ingo Molnar
2015-05-12 15:36 ` Frederic Weisbecker
2015-05-12 21:05 ` CONFIG_ISOLATION=y Chris Metcalf
2015-05-12 10:46 ` [PATCH 0/6] support "dataplane" mode for nohz_full Peter Zijlstra
2015-05-15 15:10 ` Chris Metcalf
2015-05-15 21:26 ` [PATCH v2 0/5] support "cpu_isolated" " Chris Metcalf
2015-05-15 21:27 ` [PATCH v2 1/5] nohz_full: add support for "cpu_isolated" mode Chris Metcalf
2015-05-15 21:27 ` [PATCH v2 2/5] nohz: support PR_CPU_ISOLATED_STRICT mode Chris Metcalf
2015-05-15 21:27 ` [PATCH v2 3/5] nohz: cpu_isolated strict mode configurable signal Chris Metcalf
2015-05-15 21:27 ` [PATCH v2 4/5] nohz: add cpu_isolated_debug boot flag Chris Metcalf
2015-05-15 21:27 ` [PATCH v2 5/5] nohz: cpu_isolated: allow tick to be fully disabled Chris Metcalf
2015-05-15 22:17 ` [PATCH v2 1/5] nohz_full: add support for "cpu_isolated" mode Thomas Gleixner
2015-05-28 20:38 ` Chris Metcalf
2015-06-03 15:29 ` [PATCH v3 0/5] support "cpu_isolated" mode for nohz_full Chris Metcalf
2015-06-03 15:29 ` [PATCH v3 1/5] nohz_full: add support for "cpu_isolated" mode Chris Metcalf
2015-06-03 15:29 ` [PATCH v3 2/5] nohz: support PR_CPU_ISOLATED_STRICT mode Chris Metcalf
2015-06-03 15:29 ` [PATCH v3 3/5] nohz: cpu_isolated strict mode configurable signal Chris Metcalf
2015-06-03 15:29 ` [PATCH v3 4/5] nohz: add cpu_isolated_debug boot flag Chris Metcalf
2015-06-03 15:29 ` [PATCH v3 5/5] nohz: cpu_isolated: allow tick to be fully disabled Chris Metcalf
2015-07-13 19:57 ` [PATCH v4 0/5] support "cpu_isolated" mode for nohz_full Chris Metcalf
2015-07-13 19:57 ` [PATCH v4 1/5] nohz_full: add support for "cpu_isolated" mode Chris Metcalf
2015-07-13 20:40 ` Andy Lutomirski
2015-07-13 21:01 ` Chris Metcalf
2015-07-13 21:45 ` Andy Lutomirski
2015-07-21 19:10 ` Chris Metcalf
2015-07-21 19:26 ` Andy Lutomirski
2015-07-21 20:36 ` Paul E. McKenney
2015-07-22 13:57 ` Christoph Lameter
2015-07-22 19:28 ` Paul E. McKenney
2015-07-22 20:02 ` Christoph Lameter
2015-07-24 20:21 ` Chris Metcalf
2015-07-24 20:22 ` Chris Metcalf
2015-07-24 14:03 ` Frederic Weisbecker
2015-07-24 20:19 ` Chris Metcalf
2015-07-24 13:27 ` Frederic Weisbecker
2015-07-24 20:21 ` Chris Metcalf
2015-07-13 19:57 ` [PATCH v4 2/5] nohz: support PR_CPU_ISOLATED_STRICT mode Chris Metcalf
2015-07-13 21:47 ` Andy Lutomirski
2015-07-21 19:34 ` Chris Metcalf
2015-07-21 19:42 ` Andy Lutomirski
2015-07-24 20:29 ` Chris Metcalf
2015-07-13 19:57 ` [PATCH v4 3/5] nohz: cpu_isolated strict mode configurable signal Chris Metcalf
2015-07-13 19:58 ` [PATCH v4 4/5] nohz: add cpu_isolated_debug boot flag Chris Metcalf
2015-07-13 19:58 ` [PATCH v4 5/5] nohz: cpu_isolated: allow tick to be fully disabled Chris Metcalf
2015-07-28 19:49 ` [PATCH v5 0/6] support "cpu_isolated" mode for nohz_full Chris Metcalf
2015-07-28 19:49 ` [PATCH v5 1/6] vmstat: provide a function to quiet down the diff processing Chris Metcalf
2015-07-28 19:49 ` [PATCH v5 2/6] cpu_isolated: add initial support Chris Metcalf
2015-08-12 16:00 ` Frederic Weisbecker
2015-08-12 18:22 ` Chris Metcalf
2015-08-26 15:26 ` Frederic Weisbecker
2015-08-26 15:55 ` Chris Metcalf
2015-07-28 19:49 ` [PATCH v5 3/6] cpu_isolated: support PR_CPU_ISOLATED_STRICT mode Chris Metcalf
2015-07-28 19:49 ` [PATCH v5 4/6] cpu_isolated: provide strict mode configurable signal Chris Metcalf
2015-07-28 19:49 ` [PATCH v5 5/6] cpu_isolated: add debug boot flag Chris Metcalf
2015-07-28 19:49 ` [PATCH v5 6/6] nohz: cpu_isolated: allow tick to be fully disabled Chris Metcalf
2015-08-25 19:55 ` [PATCH v6 0/6] support "task_isolated" mode for nohz_full Chris Metcalf
2015-08-25 19:55 ` [PATCH v6 1/6] vmstat: provide a function to quiet down the diff processing Chris Metcalf
2015-08-25 19:55 ` [PATCH v6 2/6] task_isolation: add initial support Chris Metcalf
2015-08-25 19:55 ` [PATCH v6 3/6] task_isolation: support PR_TASK_ISOLATION_STRICT mode Chris Metcalf
2015-08-26 10:36 ` Will Deacon
2015-08-26 15:10 ` Chris Metcalf
2015-09-02 10:13 ` Will Deacon
2015-09-02 18:38 ` [PATCH v6.2 " Chris Metcalf
2015-08-28 15:31 ` [PATCH v6.1 " Chris Metcalf
2015-08-25 19:55 ` [PATCH v6 4/6] task_isolation: provide strict mode configurable signal Chris Metcalf
2015-08-28 19:22 ` Andy Lutomirski
2015-08-25 19:55 ` [PATCH v6 5/6] task_isolation: add debug boot flag Chris Metcalf
2015-08-25 19:55 ` [PATCH v6 6/6] nohz: task_isolation: allow tick to be fully disabled Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 00/11] support "task_isolated" mode for nohz_full Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 01/11] vmstat: provide a function to quiet down the diff processing Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 02/11] task_isolation: add initial support Chris Metcalf
2015-10-01 12:14 ` Frederic Weisbecker
2015-10-01 12:18 ` Thomas Gleixner
2015-10-01 12:23 ` Frederic Weisbecker
2015-10-01 12:31 ` Thomas Gleixner
2015-10-01 17:02 ` Chris Metcalf
2015-10-01 21:20 ` Thomas Gleixner
2015-10-02 17:15 ` Chris Metcalf
2015-10-02 19:02 ` Thomas Gleixner
2015-10-01 19:25 ` Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 03/11] task_isolation: support PR_TASK_ISOLATION_STRICT mode Chris Metcalf
2015-09-28 20:51 ` Andy Lutomirski
2015-09-28 21:54 ` Chris Metcalf
2015-09-28 22:38 ` Andy Lutomirski
2015-09-29 17:35 ` Chris Metcalf
2015-09-29 17:46 ` Andy Lutomirski
2015-09-29 17:57 ` Chris Metcalf
2015-09-29 18:00 ` Andy Lutomirski
2015-10-01 19:25 ` Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 04/11] task_isolation: provide strict mode configurable signal Chris Metcalf
2015-09-28 20:54 ` Andy Lutomirski
2015-09-28 21:54 ` Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 05/11] task_isolation: add debug boot flag Chris Metcalf
2015-09-28 20:59 ` Andy Lutomirski
2015-09-28 21:55 ` Chris Metcalf
2015-09-28 22:40 ` Andy Lutomirski
2015-09-29 17:35 ` Chris Metcalf
2015-10-05 17:07 ` Luiz Capitulino
2015-10-08 0:33 ` Chris Metcalf
2015-10-08 20:28 ` Luiz Capitulino
2015-09-28 15:17 ` [PATCH v7 06/11] nohz: task_isolation: allow tick to be fully disabled Chris Metcalf
2015-09-28 20:40 ` Andy Lutomirski
2015-10-01 13:07 ` Frederic Weisbecker
2015-10-01 14:13 ` Thomas Gleixner
2015-09-28 15:17 ` [PATCH v7 07/11] arch/x86: enable task isolation functionality Chris Metcalf
2015-09-28 20:59 ` Andy Lutomirski
2015-09-28 21:57 ` Chris Metcalf
2015-09-28 22:43 ` Andy Lutomirski
2015-09-29 17:42 ` Chris Metcalf
2015-09-29 17:57 ` Andy Lutomirski
2015-09-30 20:25 ` Thomas Gleixner
2015-09-30 20:58 ` Chris Metcalf
2015-09-30 22:02 ` Thomas Gleixner
2015-09-30 22:11 ` Andy Lutomirski
2015-10-01 8:12 ` Thomas Gleixner
2015-10-01 9:08 ` Christoph Lameter
2015-10-01 10:10 ` Thomas Gleixner
2015-10-01 19:25 ` Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 08/11] arch/arm64: adopt prepare_exit_to_usermode() model from x86 Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 09/11] arch/arm64: enable task isolation functionality Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 10/11] arch/tile: adopt prepare_exit_to_usermode() model from x86 Chris Metcalf
2015-09-28 15:17 ` [PATCH v7 11/11] arch/tile: enable task isolation functionality Chris Metcalf
2015-10-20 20:35 ` [PATCH v8 00/14] support "task_isolation" mode for nohz_full Chris Metcalf
2015-10-20 20:35 ` [PATCH v8 01/14] vmstat: provide a function to quiet down the diff processing Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 02/14] vmstat: add vmstat_idle function Chris Metcalf
2015-10-20 20:45 ` Christoph Lameter
2015-10-20 20:36 ` [PATCH v8 03/14] lru_add_drain_all: factor out lru_add_drain_needed Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 04/14] task_isolation: add initial support Chris Metcalf
2015-10-20 20:56 ` Andy Lutomirski
2015-10-20 21:20 ` Chris Metcalf
2015-10-20 21:26 ` Andy Lutomirski
2015-10-21 0:29 ` Steven Rostedt
2015-10-26 20:19 ` Chris Metcalf
2015-10-26 21:13 ` Chris Metcalf
2015-10-26 20:32 ` Chris Metcalf
2015-10-21 16:12 ` Frederic Weisbecker
2015-10-27 16:40 ` Chris Metcalf
2016-01-28 16:38 ` Frederic Weisbecker
2016-02-11 19:58 ` Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 05/14] task_isolation: support PR_TASK_ISOLATION_STRICT mode Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 06/14] task_isolation: provide strict mode configurable signal Chris Metcalf
2015-10-21 0:56 ` Steven Rostedt
2015-10-21 1:30 ` Chris Metcalf
2015-10-21 1:41 ` Steven Rostedt
2015-10-21 1:42 ` Andy Lutomirski
2015-10-21 6:41 ` Gilad Ben Yossef
2015-10-21 18:53 ` Andy Lutomirski
2015-10-22 20:44 ` Chris Metcalf
2015-10-22 21:00 ` Andy Lutomirski
2015-10-27 19:37 ` Chris Metcalf
2015-10-24 9:16 ` Gilad Ben Yossef
2015-10-20 20:36 ` [PATCH v8 07/14] task_isolation: add debug boot flag Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 08/14] nohz_full: allow disabling the 1Hz minimum tick at boot Chris Metcalf
2015-10-20 21:03 ` Frederic Weisbecker
2015-10-20 21:18 ` Chris Metcalf
2015-10-21 0:59 ` Steven Rostedt
2015-10-21 6:56 ` Gilad Ben Yossef
2015-10-21 14:28 ` Christoph Lameter
2015-10-21 15:35 ` Frederic Weisbecker
2015-10-20 20:36 ` [PATCH v8 09/14] arch/x86: enable task isolation functionality Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 10/14] arch/arm64: adopt prepare_exit_to_usermode() model from x86 Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 11/14] arch/arm64: enable task isolation functionality Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 12/14] arch/tile: adopt prepare_exit_to_usermode() model from x86 Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 13/14] arch/tile: turn off timer tick for oneshot_stopped state Chris Metcalf
2015-10-20 20:36 ` [PATCH v8 14/14] arch/tile: enable task isolation functionality Chris Metcalf
2015-10-21 12:39 ` [PATCH v8 00/14] support "task_isolation" mode for nohz_full Peter Zijlstra
2015-10-22 20:31 ` Chris Metcalf
2015-10-23 2:33 ` Frederic Weisbecker
2015-10-23 8:49 ` Peter Zijlstra
2015-10-23 13:29 ` Frederic Weisbecker
2015-10-23 9:04 ` Peter Zijlstra
2015-10-23 11:52 ` Theodore Ts'o
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150509070538.GA9413@gmail.com \
--to=mingo@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=cl@linux.com \
--cc=cmetcalf@ezchip.com \
--cc=fweisbec@gmail.com \
--cc=giladb@ezchip.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=rostedt@goodmis.org \
--cc=srivatsa.bhat@linux.vnet.ibm.com \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).