From: Steven Rostedt <rostedt@goodmis.org>
To: Alexander Graf <graf@amazon.com>
Cc: <linux-kernel@vger.kernel.org>,
<linux-trace-kernel@vger.kernel.org>,
Masami Hiramatsu <mhiramat@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Andrew Morton <akpm@linux-foundation.org>,
Vincent Donnefort <vdonnefort@google.com>,
"Joel Fernandes" <joel@joelfernandes.org>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Ingo Molnar <mingo@kernel.org>,
Peter Zijlstra <peterz@infradead.org>, <suleiman@google.com>,
Thomas Gleixner <tglx@linutronix.de>,
Vineeth Pillai <vineeth@bitbyteword.org>,
Youssef Esmat <youssefesmat@google.com>,
Beau Belgrave <beaub@linux.microsoft.com>,
"Baoquan He" <bhe@redhat.com>, Borislav Petkov <bp@alien8.de>,
"Paul E. McKenney" <paulmck@kernel.org>,
David Howells <dhowells@redhat.com>,
Mike Rapoport <rppt@kernel.org>, Ard Biesheuvel <ardb@kernel.org>
Subject: Re: [PATCH v6 0/2] mm/memblock: Add "reserve_mem" to reserved named memory at boot up
Date: Mon, 17 Jun 2024 16:40:06 -0400 [thread overview]
Message-ID: <20240617164006.198b9ba3@rorschach.local.home> (raw)
In-Reply-To: <7c90c574-5cfa-40cf-bd4c-1188136cd886@amazon.com>
[-- Attachment #1: Type: text/plain, Size: 4250 bytes --]
On Mon, 17 Jun 2024 09:07:29 +0200
Alexander Graf <graf@amazon.com> wrote:
> Hey Steve,
>
>
> I believe we're talking about 2 different things :). Let me rephrase a
> bit and make a concrete example.
>
> Imagine you have passed the "reserve_mem=12M:4096:trace" kernel command
> line option. The kernel now comes up and allocates a random chunk of
> memory that - by (admittedly good) chance - may be at the same physical
> location as before. Let's assume it deemed 0x1000000 as a good offset.
Note, it's not random. It picks from the top of available memory every
time. But things can mess with it (see below).
>
> Let's now assume you're running on a UEFI system. There, you always have
> non-volatile storage available to you even in the pre-boot phase. That
> means the kernel could create a UEFI variable that says "12M:4096:trace
> -> 0x1000000". The pre-boot phase takes all these UEFI variables and
> marks them as reserved. When you finally reach your command line parsing
> logic for reserve_mem=, you can flip all reservations that were not on
> the command line back to normal memory.
>
> That way you have pretty much guaranteed persistent memory regions, even
> with KASLR changing your memory layout across boots.
>
> The nice thing is that the above is an extension of what you've already
> built: Systems with UEFI simply get better guarantees that their regions
> persist.
This could be an added feature, but it is very architecture specific,
and would likely need architecture specific updates.
>
>
> >
> >>
> >>> Requirement:
> >>>
> >>> Need a way to reserve memory that will be at a consistent location for
> >>> every boot, if the kernel and system are the same. Does not need to work
> >>> if rebooting to a different kernel, or if the system can change the
> >>> memory layout between boots.
> >>>
> >>> The reserved memory can not be an hard coded address, as the same kernel /
> >>> command line needs to run on several different machines. The picked memory
> >>> reservation just needs to be the same for a given machine, but may be
> >>
> >> With KASLR is enabled, doesn't this approach break too often to be
> >> reliable enough for the data you want to extract?
> >>
> >> Picking up the idea above, with a persistent variable we could even make
> >> KASLR avoid that reserved pstore region in its search for a viable KASLR
> >> offset.
> > I think I was hit by it once in all my testing. For our use case, the
> > few times it fails to map is not going to affect what we need this for
> > at all.
>
>
> Once is pretty good. Do you know why? Also once out of how many runs? Is
> the randomness source not as random as it should be or are the number of
> bits for KASLR maybe so few on your target architecture that the odds of
> hitting anything become low? Do these same constraints hold true outside
> of your testing environment?
So I just ran it a hundred times in a loop. I added a patch to print
the location of "_text". The loop was this:
for i in `seq 100`; do
ssh root@debiantesting-x86-64 "dmesg | grep -e 'text starts' -e 'mapped boot' >> text; grub-reboot '1>0'; sleep 0.5; reboot"
sleep 25
done
It searches dmesg for my added printk as well as the print of were the
ring buffer was loaded in physical memory.
It takes about 15 seconds to reboot, so I waited 25. The results are
attached. I found that it was consistent 76 times, which means 1 out of
4 it's not. Funny enough, it broke whenever it loaded the kernel below
0x100000000. And then it would be off by a little.
It was consistently at:
0x27d000000
And when it failed, it was at 0x27ce00000.
Note, when I used the e820 tables to do this, I never saw a failure. My
assumption is that when it is below 0x100000000, something else gets
allocated causing this to get pushed down.
As this code relies on memblock_phys_alloc() being consistent, if
something gets allocated before it differently depending on where the
kernel is, it can also move the location. A plugin to UEFI would mean
that it would need to reserve the memory, and the code here will need
to know where it is. We could always make the function reserve_mem()
global and weak so that architectures can override it.
-- Steve
[-- Attachment #2: text --]
[-- Type: application/octet-stream, Size: 15200 bytes --]
[ 0.000000] text starts at 0000000261a00000
[ 0.279601] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000264a00000
[ 0.258160] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000016d800000
[ 0.274498] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001e3200000
[ 0.321133] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000185200000
[ 0.289880] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000121a00000
[ 0.335898] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000003ca00000
[ 0.299969] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000254e00000
[ 0.257330] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001ca400000
[ 0.361895] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001a3a00000
[ 0.264609] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000221200000
[ 0.302778] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000223400000
[ 0.281307] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001e5600000
[ 0.263459] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000022dc00000
[ 0.293013] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000010dc00000
[ 0.255996] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001b0c00000
[ 0.260787] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000052400000
[ 0.257456] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000015ca00000
[ 0.271065] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000101800000
[ 0.320951] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000252400000
[ 0.307885] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000227800000
[ 0.280646] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000007800000
[ 0.252212] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000227c00000
[ 0.262189] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000025fe00000
[ 0.343060] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000223a00000
[ 0.265265] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001afa00000
[ 0.324491] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000151400000
[ 0.275907] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000136e00000
[ 0.296053] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001a3200000
[ 0.256108] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000278400000
[ 0.254019] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000061800000
[ 0.253513] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 00000001b5000000
[ 0.283077] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000029600000
[ 0.253972] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000005d600000
[ 0.258085] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 00000001ade00000
[ 0.263642] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001d0400000
[ 0.254616] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000222200000
[ 0.275135] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000002f200000
[ 0.257120] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000014b200000
[ 0.253358] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000107200000
[ 0.277631] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000185000000
[ 0.277916] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001f5600000
[ 0.267432] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000006fa00000
[ 0.254557] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000118000000
[ 0.259598] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000018b200000
[ 0.277552] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001b1400000
[ 0.248468] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000056400000
[ 0.253134] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000001fe00000
[ 0.309512] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 00000001e6400000
[ 0.262389] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000016e600000
[ 0.261098] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000184600000
[ 0.246911] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000275a00000
[ 0.245947] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000023e200000
[ 0.243659] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000103800000
[ 0.245082] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000004e000000
[ 0.311655] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000022dc00000
[ 0.303337] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000226800000
[ 0.248418] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001e0800000
[ 0.248551] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000115c00000
[ 0.254711] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001bba00000
[ 0.304443] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000010c200000
[ 0.250813] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000142a00000
[ 0.250088] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000012fa00000
[ 0.268560] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000140000000
[ 0.252298] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001adc00000
[ 0.251261] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000012200000
[ 0.248004] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000010ce00000
[ 0.250743] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000005b400000
[ 0.313805] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 00000001d9200000
[ 0.000000] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000029400000
[ 0.242129] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000060c00000
[ 0.303373] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000000c200000
[ 0.290952] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000250200000
[ 0.310198] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000003d600000
[ 0.246804] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000019fe00000
[ 0.265477] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000175000000
[ 0.255366] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000133600000
[ 0.270127] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000011d600000
[ 0.281918] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000024a00000
[ 0.256186] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000165a00000
[ 0.257470] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000003dc00000
[ 0.263671] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000102600000
[ 0.254288] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 00000001ce200000
[ 0.335659] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000018ac00000
[ 0.252010] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000074a00000
[ 0.254337] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 00000001ece00000
[ 0.311124] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000004d400000
[ 0.260534] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000013c00000
[ 0.295823] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 000000005ec00000
[ 0.271064] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000166c00000
[ 0.255484] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000025fa00000
[ 0.257275] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000021cc00000
[ 0.254915] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000239400000
[ 0.283687] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000018f000000
[ 0.258722] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000275800000
[ 0.281587] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000157400000
[ 0.256042] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000215c00000
[ 0.000000] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 0000000115400000
[ 0.243701] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
[ 0.000000] text starts at 000000004f400000
[ 0.319405] Tracing: mapped boot instance boot_mapped at physical memory 0x27ce00000 of size 0xc00000
[ 0.000000] text starts at 0000000267800000
[ 0.307464] Tracing: mapped boot instance boot_mapped at physical memory 0x27d000000 of size 0xc00000
next prev parent reply other threads:[~2024-06-17 20:40 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-13 15:55 [PATCH v6 0/2] mm/memblock: Add "reserve_mem" to reserved named memory at boot up Steven Rostedt
2024-06-13 15:55 ` [PATCH v6 1/2] " Steven Rostedt
2024-06-18 12:55 ` Zhengyejian
2024-06-18 16:06 ` Steven Rostedt
2024-06-19 6:42 ` Mike Rapoport
2024-06-13 15:55 ` [PATCH v6 2/2] pstore/ramoops: Add ramoops.mem_name= command line option Steven Rostedt
2024-06-13 18:19 ` Guilherme G. Piccoli
2024-06-13 18:42 ` Steven Rostedt
2024-06-13 18:51 ` Guilherme G. Piccoli
2024-06-13 16:54 ` [PATCH v6 0/2] mm/memblock: Add "reserve_mem" to reserved named memory at boot up Alexander Graf
2024-06-13 17:12 ` Steven Rostedt
2024-06-17 7:07 ` Alexander Graf
2024-06-17 20:40 ` Steven Rostedt [this message]
2024-06-17 21:01 ` Alexander Graf
2024-06-17 21:18 ` Steven Rostedt
2024-06-18 10:21 ` Ard Biesheuvel
2024-06-18 11:47 ` Alexander Graf
2024-06-18 15:43 ` Steven Rostedt
2024-06-19 8:02 ` Mike Rapoport
2024-06-19 21:53 ` Mike Rapoport
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240617164006.198b9ba3@rorschach.local.home \
--to=rostedt@goodmis.org \
--cc=akpm@linux-foundation.org \
--cc=ardb@kernel.org \
--cc=beaub@linux.microsoft.com \
--cc=bhe@redhat.com \
--cc=bp@alien8.de \
--cc=bristot@redhat.com \
--cc=dhowells@redhat.com \
--cc=graf@amazon.com \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mathieu.desnoyers@efficios.com \
--cc=mhiramat@kernel.org \
--cc=mingo@kernel.org \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=rppt@kernel.org \
--cc=suleiman@google.com \
--cc=tglx@linutronix.de \
--cc=vdonnefort@google.com \
--cc=vineeth@bitbyteword.org \
--cc=youssefesmat@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).