* [RFC PATCH 0/3] powerpc: initial support for Kexec HandOver (KHO)
@ 2026-08-21 10:56 Sourabh Jain
2026-08-21 10:56 ` [RFC PATCH 1/3] kernel/liveupdate: make KEXEC_HANDOVER depend on KEXEC_FILE instead of selecting it Sourabh Jain
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Sourabh Jain @ 2026-08-21 10:56 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sourabh Jain, Aditya Gupta, Alexander Graf, Andrew Morton,
Baoquan He, Christophe Leroy (CS GROUP), Hari Bathini,
Madhavan Srinivasan, Mahesh Salgaonkar, Michael Ellerman,
Mike Rapoport, Nicholas Piggin, Pasha Tatashin, Pratyush Yadav,
Ritesh Harjani (IBM), Shivang Upadhyay, Shrikanth Hegde, kexec,
linux-kernel
This series adds initial powerpc support for Kexec HandOver (KHO),
allowing state to be preserved across kexec on powerpc. Posting as RFC
because some of the patches are workarounds for issues I ran into, and
I'd like input on how best to address them.
Patch 1 is a small Kconfig fix needed to unblock patch 2. It won't be
needed once the ordering problem described under patch 2 is
addressed properly.
Patch 2 wires up ARCH_SUPPORTS_KEXEC_HANDOVER and adds the code to
retrieve the KHO FDT and scratch region from /chosen on boot. On
powerpc, crashkernel memory is reserved very early, from arch-specific
code, while KHO's scratch region is reserved later, from generic code
(kho_memory_init()). Crashkernel reservation always runs first, so by
the time KHO reserves its scratch region, not enough memory is left
and the reservation fails. Until scratch reservation can be moved
ahead of crashkernel reservation, this patch makes KHO and CRASH_DUMP
mutually exclusive on powerpc as a stopgap. The failure scenario and
the reasoning behind this restriction are described in the commit
message with an example.
Patch 3 excludes TRANSPARENT_HUGEPAGE and HUGETLB_PAGE from
ARCH_SUPPORTS_KEXEC_HANDOVER, since KHO's static_assert() on
SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES fails to build
when either is enabled -- pageblock_order isn't a compile-time
constant on powerpc in that configuration.
With all three patches applied, CONFIG_TEST_KEXEC_HANDOVER passes on
powerpc.
Open questions I'd appreciate feedback on:
- How should KHO scratch reservation and powerpc's crashkernel
reservation be ordered so both can be enabled together, given that
scratch reservation currently happens in generic code
(kho_memory_init())?
- Excluding THP/HUGETLB_PAGE is a workaround for the build failure.
Looking for input on how to handle CMA_MIN_ALIGNMENT_BYTES when
pageblock_order is non-constant.
Cc: Aditya Gupta <adityag@linux.ibm.com>
Cc: Alexander Graf <graf@amazon.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shivang Upadhyay <shivangu@linux.ibm.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Sourabh Jain <sourabhjain@linux.ibm.com>
Cc: kexec@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Sourabh Jain (3):
kernel/liveupdate: make KEXEC_HANDOVER depend on KEXEC_FILE instead of
selecting it
powerpc: add support for Kexec HandOver (KHO)
powerpc/kho: exclude THP and HUGETLB_PAGE
arch/powerpc/Kconfig | 6 ++++++
arch/powerpc/kernel/setup-common.c | 33 ++++++++++++++++++++++++++++++
kernel/liveupdate/Kconfig | 2 +-
3 files changed, 40 insertions(+), 1 deletion(-)
--
2.55.0
^ permalink raw reply [flat|nested] 8+ messages in thread* [RFC PATCH 1/3] kernel/liveupdate: make KEXEC_HANDOVER depend on KEXEC_FILE instead of selecting it 2026-08-21 10:56 [RFC PATCH 0/3] powerpc: initial support for Kexec HandOver (KHO) Sourabh Jain @ 2026-08-21 10:56 ` Sourabh Jain 2026-08-21 10:56 ` [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO) Sourabh Jain 2026-08-21 10:56 ` [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE Sourabh Jain 2 siblings, 0 replies; 8+ messages in thread From: Sourabh Jain @ 2026-08-21 10:56 UTC (permalink / raw) To: linuxppc-dev Cc: Sourabh Jain, Aditya Gupta, Alexander Graf, Andrew Morton, Baoquan He, Christophe Leroy (CS GROUP), Hari Bathini, Madhavan Srinivasan, Mahesh Salgaonkar, Michael Ellerman, Mike Rapoport, Nicholas Piggin, Pasha Tatashin, Pratyush Yadav, Ritesh Harjani (IBM), Shivang Upadhyay, Shrikanth Hegde, kexec, linux-kernel An upcoming patch makes ARCH_SUPPORTS_KEXEC_HANDOVER depend on CRASH_DUMP on powerpc: config ARCH_SUPPORTS_KEXEC_HANDOVER def_bool y depends on PPC64 depends on !CRASH_DUMP That triggers a recursive dependency error: error: recursive dependency detected! symbol KEXEC_HANDOVER depends on ARCH_SUPPORTS_KEXEC_HANDOVER symbol ARCH_SUPPORTS_KEXEC_HANDOVER depends on CRASH_DUMP symbol CRASH_DUMP depends on KEXEC_CORE symbol KEXEC_CORE is selected by KEXEC_FILE symbol KEXEC_FILE is selected by KEXEC_HANDOVER KEXEC_HANDOVER and KEXEC_CORE end up depending on each other when ARCH_SUPPORTS_KEXEC_HANDOVER depends on CRASH_DUMP. This creates a recursive Kconfig dependency. Change KEXEC_FILE from select to depends on to break this cycle. This means KEXEC_FILE must be enabled explicitly instead of being enabled automatically by KEXEC_HANDOVER. Keep this change until the powerpc-specific KHO and CRASH_DUMP dependency is resolved. Cc: Aditya Gupta <adityag@linux.ibm.com> Cc: Alexander Graf <graf@amazon.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org> Cc: Hari Bathini <hbathini@linux.ibm.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Mike Rapoport <rppt@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Pasha Tatashin <pasha.tatashin@soleen.com> Cc: Pratyush Yadav <pratyush@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shivang Upadhyay <shivangu@linux.ibm.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: kexec@lists.infradead.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com> --- kernel/liveupdate/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/liveupdate/Kconfig b/kernel/liveupdate/Kconfig index c13af38ba23a..22e317dc1e1a 100644 --- a/kernel/liveupdate/Kconfig +++ b/kernel/liveupdate/Kconfig @@ -6,7 +6,7 @@ config KEXEC_HANDOVER bool "kexec handover" depends on ARCH_SUPPORTS_KEXEC_HANDOVER && ARCH_SUPPORTS_KEXEC_FILE select MEMBLOCK_KHO_SCRATCH - select KEXEC_FILE + depends on KEXEC_FILE select LIBFDT select CMA help -- 2.55.0 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO) 2026-08-21 10:56 [RFC PATCH 0/3] powerpc: initial support for Kexec HandOver (KHO) Sourabh Jain 2026-08-21 10:56 ` [RFC PATCH 1/3] kernel/liveupdate: make KEXEC_HANDOVER depend on KEXEC_FILE instead of selecting it Sourabh Jain @ 2026-08-21 10:56 ` Sourabh Jain 2026-08-21 11:56 ` Pratyush Yadav 2026-08-21 10:56 ` [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE Sourabh Jain 2 siblings, 1 reply; 8+ messages in thread From: Sourabh Jain @ 2026-08-21 10:56 UTC (permalink / raw) To: linuxppc-dev Cc: Sourabh Jain, Aditya Gupta, Alexander Graf, Andrew Morton, Baoquan He, Christophe Leroy (CS GROUP), Hari Bathini, Madhavan Srinivasan, Mahesh Salgaonkar, Michael Ellerman, Mike Rapoport, Nicholas Piggin, Pasha Tatashin, Pratyush Yadav, Ritesh Harjani (IBM), Shivang Upadhyay, Shrikanth Hegde, kexec, linux-kernel Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on powerpc. Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE. On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch" properties from /chosen and pass them to kho_populate(). This lets a kernel booted via KHO kexec recover the FDT and scratch region left behind by the previous kernel. The call is placed early in setup_arch(), before unflatten_device_tree(). Open issues: ============ This patch also adds "depends on !CRASH_DUMP" to ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering conflict between crashkernel reservation and KHO scratch reservation on powerpc. Crashkernel memory is reserved very early in boot, from arch-specific code: head.S -> early_setup() -> early_init_devtree() -> arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch region is reserved later, from generic code: start_kernel() -> mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel memory is always reserved first. This ordering causes a real failure. In the common case, crashkernel reservation on powerpc starts at a 512M offset (the exact offset can vary, but 512M is typical). So with crashkernel=3G, the reservation occupies memory from 512M up to 3.5G -- roughly 75% of the entire low 4G area. Since crashkernel reservation always happens first, that 3G is already committed by the time kho_memory_init() runs. It then tries to reserve a low scratch region sized at 200% of whatever is already reserved below 4G. With ~75% of that 4G area already taken by crashkernel memory, 200% of that easily exceeds the remaining space -- and since the low scratch region is itself capped at 4G, there's no room left to fit it. The reservation fails. To work around this and get KHO working on powerpc, this patch: 1. Makes KHO usable on powerpc only when CRASH_DUMP is disabled. 2. Calls kho_populate() from setup_arch(), so it runs before kho_memory_init() reserves the scratch region. The real fix would be to reserve the KHO scratch region before crashkernel memory instead of after. But scratch reservation happens in generic code (kho_memory_init(), called from mm_core_init()), so this isn't something powerpc can address on its own -- it needs discussion on how to influence the ordering between generic scratch reservation and arch-specific crashkernel reservation. This patch doesn't attempt that; it's meant as a starting point for that discussion. Together with patch 3, this gets CONFIG_TEST_KEXEC_HANDOVER passing on powerpc. Cc: Aditya Gupta <adityag@linux.ibm.com> Cc: Alexander Graf <graf@amazon.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org> Cc: Hari Bathini <hbathini@linux.ibm.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Mike Rapoport <rppt@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Pasha Tatashin <pasha.tatashin@soleen.com> Cc: Pratyush Yadav <pratyush@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shivang Upadhyay <shivangu@linux.ibm.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: kexec@lists.infradead.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com> --- arch/powerpc/Kconfig | 5 +++++ arch/powerpc/kernel/setup-common.c | 33 ++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 2580e27e4328..61350d3e7a19 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -716,6 +716,11 @@ config ARCH_SELECTS_CRASH_DUMP depends on CRASH_DUMP select RELOCATABLE if PPC64 || 44x || PPC_85xx +config ARCH_SUPPORTS_KEXEC_HANDOVER + def_bool y + depends on PPC64 + depends on !CRASH_DUMP + config ARCH_SUPPORTS_CRASH_HOTPLUG def_bool y depends on PPC64 diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c index 4afaba19b586..1fee743abdf2 100644 --- a/arch/powerpc/kernel/setup-common.c +++ b/arch/powerpc/kernel/setup-common.c @@ -35,6 +35,8 @@ #include <linux/of_irq.h> #include <linux/hugetlb.h> #include <linux/pgtable.h> +#include <linux/libfdt.h> +#include <linux/kexec_handover.h> #include <asm/io.h> #include <asm/paca.h> #include <asm/processor.h> @@ -910,6 +912,33 @@ static void __init smp_setup_pacas(void) } #endif +#ifdef CONFIG_PPC64 +static void __init init_kho(const void *fdt) +{ + unsigned long node; + u64 fdt_start, fdt_size, scratch_start, scratch_size; + + if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER)) + return; + + /* Find and verify the /chosen node, same as early_init_dt_scan_chosen() does */ + node = fdt_path_offset(fdt, "/chosen"); + if ((long)node < 0) + node = fdt_path_offset(fdt, "/chosen@0"); + if ((long)node < 0) + return; + + if (!of_flat_dt_get_addr_size(node, "linux,kho-fdt", + &fdt_start, &fdt_size)) + return; + if (!of_flat_dt_get_addr_size(node, "linux,kho-scratch", + &scratch_start, &scratch_size)) + return; + + kho_populate(fdt_start, fdt_size, scratch_start, scratch_size); +} +#endif + /* * Called into from start_kernel this initializes memblock, which is used * to manage page allocation until mem_init is called. @@ -923,6 +952,10 @@ void __init setup_arch(char **cmdline_p) /* Set a half-reasonable default so udelay does something sensible */ loops_per_jiffy = 500000000 / HZ; +#ifdef CONFIG_PPC64 + init_kho(initial_boot_params); +#endif + /* Unflatten the device-tree passed by prom_init or kexec */ unflatten_device_tree(); -- 2.55.0 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO) 2026-08-21 10:56 ` [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO) Sourabh Jain @ 2026-08-21 11:56 ` Pratyush Yadav 2026-08-23 15:41 ` Sourabh Jain 0 siblings, 1 reply; 8+ messages in thread From: Pratyush Yadav @ 2026-08-21 11:56 UTC (permalink / raw) To: Sourabh Jain Cc: linuxppc-dev, Aditya Gupta, Alexander Graf, Andrew Morton, Baoquan He, Christophe Leroy (CS GROUP), Hari Bathini, Madhavan Srinivasan, Mahesh Salgaonkar, Michael Ellerman, Mike Rapoport, Nicholas Piggin, Pasha Tatashin, Pratyush Yadav, Ritesh Harjani (IBM), Shivang Upadhyay, Shrikanth Hegde, kexec, linux-kernel On Fri, Aug 21 2026, Sourabh Jain wrote: > Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on > powerpc. > > Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing > pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE. > > On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch" > properties from /chosen and pass them to kho_populate(). This lets a > kernel booted via KHO kexec recover the FDT and scratch region left > behind by the previous kernel. The call is placed early in > setup_arch(), before unflatten_device_tree(). > > Open issues: > ============ > > This patch also adds "depends on !CRASH_DUMP" to > ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering > conflict between crashkernel reservation and KHO scratch reservation > on powerpc. > > Crashkernel memory is reserved very early in boot, from arch-specific > code: head.S -> early_setup() -> early_init_devtree() -> > arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch > region is reserved later, from generic code: start_kernel() -> > mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel > memory is always reserved first. > > This ordering causes a real failure. In the common case, crashkernel > reservation on powerpc starts at a 512M offset (the exact offset can > vary, but 512M is typical). So with crashkernel=3G, the reservation > occupies memory from 512M up to 3.5G -- roughly 75% of the entire low > 4G area. > > Since crashkernel reservation always happens first, that 3G is > already committed by the time kho_memory_init() runs. It then tries > to reserve a low scratch region sized at 200% of whatever is already > reserved below 4G. With ~75% of that 4G area already taken by > crashkernel memory, 200% of that easily exceeds the remaining space > -- and since the low scratch region is itself capped at 4G, there's > no room left to fit it. The reservation fails. crashkernel has the variant "crashkernel=size[KMG],high", which ensures memory is allocated above 4G. Unless powerpc has some requirement for strictly having the crashkernel below 4G, I think it will make a lot of sense to enable support for this feature. So KHO users can specify this to get crashkernel working with KHO. Powerpc doesn't support this right now, but from a quick skim of the code, I think it should be simple enough. From arch_reserve_crashkernel() you just need to pass a bool * to parse_crashkernel(), and then pass the result to reserve_crashkernel_generic(). Solving the ordering of crash reservations and KHO is tricky and comes with some difficult tradeoffs. Allocating crash from highmem should be a lot simpler. And on that note, I don't think you should do a depends on !CRASH_DUMP. Even when CONFIG_KEXEC_HANDOVER is enabled, KHO isn't on by default (well, unless KEXEC_HANDOVER_ENABLE_DEFAULT is set). You need to enable it via cmdline. So it is entirely possible for people using KHO on PPC to not use crash and vice versa. This decision can be made at deployment time, not at compile time. > > To work around this and get KHO working on powerpc, this patch: > > 1. Makes KHO usable on powerpc only when CRASH_DUMP is disabled. > 2. Calls kho_populate() from setup_arch(), so it runs before > kho_memory_init() reserves the scratch region. > > The real fix would be to reserve the KHO scratch region before > crashkernel memory instead of after. But scratch reservation happens > in generic code (kho_memory_init(), called from mm_core_init()), so > this isn't something powerpc can address on its own -- it needs > discussion on how to influence the ordering between generic scratch > reservation and arch-specific crashkernel reservation. This patch > doesn't attempt that; it's meant as a starting point for that > discussion. > [...] > diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig > index 2580e27e4328..61350d3e7a19 100644 > --- a/arch/powerpc/Kconfig > +++ b/arch/powerpc/Kconfig > @@ -716,6 +716,11 @@ config ARCH_SELECTS_CRASH_DUMP > depends on CRASH_DUMP > select RELOCATABLE if PPC64 || 44x || PPC_85xx > > +config ARCH_SUPPORTS_KEXEC_HANDOVER > + def_bool y > + depends on PPC64 > + depends on !CRASH_DUMP > + > config ARCH_SUPPORTS_CRASH_HOTPLUG > def_bool y > depends on PPC64 > diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c > index 4afaba19b586..1fee743abdf2 100644 > --- a/arch/powerpc/kernel/setup-common.c > +++ b/arch/powerpc/kernel/setup-common.c [...] > } > #endif > > +#ifdef CONFIG_PPC64 > +static void __init init_kho(const void *fdt) > +{ > + unsigned long node; > + u64 fdt_start, fdt_size, scratch_start, scratch_size; > + > + if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER)) > + return; > + > + /* Find and verify the /chosen node, same as early_init_dt_scan_chosen() does */ > + node = fdt_path_offset(fdt, "/chosen"); > + if ((long)node < 0) > + node = fdt_path_offset(fdt, "/chosen@0"); > + if ((long)node < 0) > + return; > + > + if (!of_flat_dt_get_addr_size(node, "linux,kho-fdt", > + &fdt_start, &fdt_size)) > + return; > + if (!of_flat_dt_get_addr_size(node, "linux,kho-scratch", > + &scratch_start, &scratch_size)) > + return; > + > + kho_populate(fdt_start, fdt_size, scratch_start, scratch_size); > +} This looks pretty much a duplicate of early_init_dt_check_kho(). On arm64 this is called via early_init_dt_scan(). But from a quick search I don't see powerpc calling it. Would it make sense to call this function (or early_init_dt_scan_nodes()) for powerpc? If not, I think it would be a better idea to expose early_init_dt_check_kho() and call it from powerpc setup_arch() instead of duplicating the logic. > +#endif > + > /* > * Called into from start_kernel this initializes memblock, which is used > * to manage page allocation until mem_init is called. [...] -- Regards, Pratyush Yadav ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO) 2026-08-21 11:56 ` Pratyush Yadav @ 2026-08-23 15:41 ` Sourabh Jain 0 siblings, 0 replies; 8+ messages in thread From: Sourabh Jain @ 2026-08-23 15:41 UTC (permalink / raw) To: Pratyush Yadav Cc: linuxppc-dev, Aditya Gupta, Alexander Graf, Andrew Morton, Baoquan He, Christophe Leroy (CS GROUP), Hari Bathini, Madhavan Srinivasan, Mahesh Salgaonkar, Michael Ellerman, Mike Rapoport, Nicholas Piggin, Pasha Tatashin, Ritesh Harjani (IBM), Shivang Upadhyay, Shrikanth Hegde, kexec, linux-kernel On 21/08/26 17:26, Pratyush Yadav wrote: > On Fri, Aug 21 2026, Sourabh Jain wrote: > >> Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on >> powerpc. >> >> Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing >> pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE. >> >> On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch" >> properties from /chosen and pass them to kho_populate(). This lets a >> kernel booted via KHO kexec recover the FDT and scratch region left >> behind by the previous kernel. The call is placed early in >> setup_arch(), before unflatten_device_tree(). >> >> Open issues: >> ============ >> >> This patch also adds "depends on !CRASH_DUMP" to >> ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering >> conflict between crashkernel reservation and KHO scratch reservation >> on powerpc. >> >> Crashkernel memory is reserved very early in boot, from arch-specific >> code: head.S -> early_setup() -> early_init_devtree() -> >> arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch >> region is reserved later, from generic code: start_kernel() -> >> mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel >> memory is always reserved first. >> >> This ordering causes a real failure. In the common case, crashkernel >> reservation on powerpc starts at a 512M offset (the exact offset can >> vary, but 512M is typical). So with crashkernel=3G, the reservation >> occupies memory from 512M up to 3.5G -- roughly 75% of the entire low >> 4G area. >> >> Since crashkernel reservation always happens first, that 3G is >> already committed by the time kho_memory_init() runs. It then tries >> to reserve a low scratch region sized at 200% of whatever is already >> reserved below 4G. With ~75% of that 4G area already taken by >> crashkernel memory, 200% of that easily exceeds the remaining space >> -- and since the low scratch region is itself capped at 4G, there's >> no room left to fit it. The reservation fails. > crashkernel has the variant "crashkernel=size[KMG],high", which ensures > memory is allocated above 4G. Unless powerpc has some requirement for > strictly having the crashkernel below 4G, I think it will make a lot of > sense to enable support for this feature. So KHO users can specify this > to get crashkernel working with KHO. I agree that this is one way to work around the low-memory reservation problem. However, there are a few things that come into play here: 1. On powerpc, the crashkernel reservation can go up to 64 GB for kdump. With the current default scratch memory reservation policy, this could result in reserving up to 256 GB of scratch memory: 200% for the high-memory reservation and another 200% for per-node memory. For fadump, which is the powerpc-specific memory dump capture mechanism, the crashkernel reservation can go up to 180 GB. In this case, we could end up reserving up to 720 GB of scratch memory, which is too much. I agree that users can tune this, but I think the default scale should be more reasonable for powerpc. 2. Fadump also uses the crashkernel kernel command-line argument, but its reservation policy is different from kdump. The crashkernel base address starts after the memory needed for fadump. For example, with crashkernel=3G, the base address would be 3 GB, and the crashkernel reservation would be from 3 GB to 6 GB. So, depending on the crashkernel size, the reservation may or may not fall within low memory. Also, fadump does not support crashkernel=xxM,high. 3. I do have a patch [1] to support high crashkernel reservations with kdump, but this would not work with Hash MMU. With Hash MMU, the kernel image is constrained to low memory, whereas with crashkernel=,high, all segments would be loaded into high memory. So, while supporting crashkernel=,high can help address the low-memory reservation issue for kdump, I think there are still some powerpc-specific constraints to consider. I would like to explore whether we can fix the ordering between crashkernel and scratch memory reservations, so that we can avoid unnecessarily large scratch memory reservations and address some of the other constraints mentioned above. Please share your thoughts. > > Powerpc doesn't support this right now, but from a quick skim of the > code, I think it should be simple enough. I have patch series under review for the same: [1] https://lore.kernel.org/all/20260708143357.673251-1-sourabhjain@linux.ibm.com/ > From > arch_reserve_crashkernel() you just need to pass a bool * to > parse_crashkernel(), and then pass the result to > reserve_crashkernel_generic(). Due to some architecture-specific dependencies (such as RTAS), booting the kernel from above 4G with support for high crashkernel is not as straightforward as on other architectures. Patch 2/4 in [1] has the details. > > Solving the ordering of crash reservations and KHO is tricky and comes > with some difficult tradeoffs. Allocating crash from highmem should be a > lot simpler. Could you please elaborate a bit on what makes the ordering tricky and what the main tradeoffs are between crash reservations and KHO? It would help me better understand the concerns here. > > And on that note, I don't think you should do a depends on !CRASH_DUMP. > Even when CONFIG_KEXEC_HANDOVER is enabled, KHO isn't on by default > (well, unless KEXEC_HANDOVER_ENABLE_DEFAULT is set). You need to enable > it via cmdline. So it is entirely possible for people using KHO on PPC > to not use crash and vice versa. This decision can be made at deployment > time, not at compile time. Agree. depends on !CRASH_DUMP is temporary and will be removed once we settle the crashkernel reservation and scratch region handling. > >> To work around this and get KHO working on powerpc, this patch: >> >> 1. Makes KHO usable on powerpc only when CRASH_DUMP is disabled. >> 2. Calls kho_populate() from setup_arch(), so it runs before >> kho_memory_init() reserves the scratch region. >> >> The real fix would be to reserve the KHO scratch region before >> crashkernel memory instead of after. But scratch reservation happens >> in generic code (kho_memory_init(), called from mm_core_init()), so >> this isn't something powerpc can address on its own -- it needs >> discussion on how to influence the ordering between generic scratch >> reservation and arch-specific crashkernel reservation. This patch >> doesn't attempt that; it's meant as a starting point for that >> discussion. >> > [...] >> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig >> index 2580e27e4328..61350d3e7a19 100644 >> --- a/arch/powerpc/Kconfig >> +++ b/arch/powerpc/Kconfig >> @@ -716,6 +716,11 @@ config ARCH_SELECTS_CRASH_DUMP >> depends on CRASH_DUMP >> select RELOCATABLE if PPC64 || 44x || PPC_85xx >> >> +config ARCH_SUPPORTS_KEXEC_HANDOVER >> + def_bool y >> + depends on PPC64 >> + depends on !CRASH_DUMP >> + >> config ARCH_SUPPORTS_CRASH_HOTPLUG >> def_bool y >> depends on PPC64 >> diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c >> index 4afaba19b586..1fee743abdf2 100644 >> --- a/arch/powerpc/kernel/setup-common.c >> +++ b/arch/powerpc/kernel/setup-common.c > [...] >> } >> #endif >> >> +#ifdef CONFIG_PPC64 >> +static void __init init_kho(const void *fdt) >> +{ >> + unsigned long node; >> + u64 fdt_start, fdt_size, scratch_start, scratch_size; >> + >> + if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER)) >> + return; >> + >> + /* Find and verify the /chosen node, same as early_init_dt_scan_chosen() does */ >> + node = fdt_path_offset(fdt, "/chosen"); >> + if ((long)node < 0) >> + node = fdt_path_offset(fdt, "/chosen@0"); >> + if ((long)node < 0) >> + return; >> + >> + if (!of_flat_dt_get_addr_size(node, "linux,kho-fdt", >> + &fdt_start, &fdt_size)) >> + return; >> + if (!of_flat_dt_get_addr_size(node, "linux,kho-scratch", >> + &scratch_start, &scratch_size)) >> + return; >> + >> + kho_populate(fdt_start, fdt_size, scratch_start, scratch_size); >> +} > This looks pretty much a duplicate of early_init_dt_check_kho(). On > arm64 this is called via early_init_dt_scan(). But from a quick search I > don't see powerpc calling it. Yes it is not called on powerpc. > > Would it make sense to call this function (or > early_init_dt_scan_nodes()) for powerpc? > > If not, I think it would be a better idea to expose > early_init_dt_check_kho() and call it from powerpc setup_arch() instead > of duplicating the logic. Agree they are identical. I would prefer making early_init_dt_check_kho() as public function and call from arch specific code. Thanks for the review. - Sourabh Jain > >> +#endif >> + >> /* >> * Called into from start_kernel this initializes memblock, which is used >> * to manage page allocation until mem_init is called. > [...] > ^ permalink raw reply [flat|nested] 8+ messages in thread
* [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE 2026-08-21 10:56 [RFC PATCH 0/3] powerpc: initial support for Kexec HandOver (KHO) Sourabh Jain 2026-08-21 10:56 ` [RFC PATCH 1/3] kernel/liveupdate: make KEXEC_HANDOVER depend on KEXEC_FILE instead of selecting it Sourabh Jain 2026-08-21 10:56 ` [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO) Sourabh Jain @ 2026-08-21 10:56 ` Sourabh Jain 2026-08-21 11:34 ` Pratyush Yadav 2 siblings, 1 reply; 8+ messages in thread From: Sourabh Jain @ 2026-08-21 10:56 UTC (permalink / raw) To: linuxppc-dev Cc: Sourabh Jain, Aditya Gupta, Alexander Graf, Andrew Morton, Baoquan He, Christophe Leroy (CS GROUP), Hari Bathini, Madhavan Srinivasan, Mahesh Salgaonkar, Michael Ellerman, Mike Rapoport, Nicholas Piggin, Pasha Tatashin, Pratyush Yadav, Ritesh Harjani (IBM), Shivang Upadhyay, Shrikanth Hegde, kexec, linux-kernel Enabling CONFIG_KEXEC_HANDOVER together with CONFIG_TRANSPARENT_HUGEPAGE or CONFIG_HUGETLB_PAGE on powerpc fails to build: kernel/liveupdate/kexec_handover.c:49:39: error: expression in static assertion is not constant 49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES); ./include/linux/build_bug.h:80:56: note: in definition of macro '__static_assert' 80 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) | ^~~~ kernel/liveupdate/kexec_handover.c:49:1: note: in expansion of macro 'static_assert' 49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES); CMA_MIN_ALIGNMENT_BYTES is derived from pageblock_order, which on powerpc is not a compile-time constant when THP or HUGETLB_PAGE is enabled -- powerpc can determine huge page size at runtime in that configuration, so pageblock_order becomes a variable rather than a constant expression. KHO's static_assert() on SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES then fails to compile, since it requires both sides to be constant expressions. Until CMA_MIN_ALIGNMENT_BYTES (or KHO's use of it) is made to handle the non-constant pageblock_order case on powerpc, exclude TRANSPARENT_HUGEPAGE and HUGETLB_PAGE from ARCH_SUPPORTS_KEXEC_HANDOVER so the two can't be enabled together and hit this build failure. Cc: Aditya Gupta <adityag@linux.ibm.com> Cc: Alexander Graf <graf@amazon.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org> Cc: Hari Bathini <hbathini@linux.ibm.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Mike Rapoport <rppt@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Pasha Tatashin <pasha.tatashin@soleen.com> Cc: Pratyush Yadav <pratyush@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shivang Upadhyay <shivangu@linux.ibm.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: kexec@lists.infradead.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com> --- arch/powerpc/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 61350d3e7a19..f68b9fc883ba 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -720,6 +720,7 @@ config ARCH_SUPPORTS_KEXEC_HANDOVER def_bool y depends on PPC64 depends on !CRASH_DUMP + depends on !TRANSPARENT_HUGEPAGE && !HUGETLB_PAGE config ARCH_SUPPORTS_CRASH_HOTPLUG def_bool y -- 2.55.0 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE 2026-08-21 10:56 ` [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE Sourabh Jain @ 2026-08-21 11:34 ` Pratyush Yadav 2026-08-23 13:52 ` Sourabh Jain 0 siblings, 1 reply; 8+ messages in thread From: Pratyush Yadav @ 2026-08-21 11:34 UTC (permalink / raw) To: Sourabh Jain Cc: linuxppc-dev, Aditya Gupta, Alexander Graf, Andrew Morton, Baoquan He, Christophe Leroy (CS GROUP), Hari Bathini, Madhavan Srinivasan, Mahesh Salgaonkar, Michael Ellerman, Mike Rapoport, Nicholas Piggin, Pasha Tatashin, Pratyush Yadav, Ritesh Harjani (IBM), Shivang Upadhyay, Shrikanth Hegde, kexec, linux-kernel, Michal Clapinski On Fri, Aug 21 2026, Sourabh Jain wrote: > Enabling CONFIG_KEXEC_HANDOVER together with CONFIG_TRANSPARENT_HUGEPAGE > or CONFIG_HUGETLB_PAGE on powerpc fails to build: > > kernel/liveupdate/kexec_handover.c:49:39: error: expression in static assertion is not constant > 49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES); > ./include/linux/build_bug.h:80:56: note: in definition of macro '__static_assert' > 80 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) > | ^~~~ > kernel/liveupdate/kexec_handover.c:49:1: note: in expansion of macro 'static_assert' > 49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES); > > CMA_MIN_ALIGNMENT_BYTES is derived from pageblock_order, which on > powerpc is not a compile-time constant when THP or HUGETLB_PAGE is > enabled -- powerpc can determine huge page size at runtime in that > configuration, so pageblock_order becomes a variable rather than a > constant expression. KHO's static_assert() on > SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES then fails to > compile, since it requires both sides to be constant expressions. > > Until CMA_MIN_ALIGNMENT_BYTES (or KHO's use of it) is made to handle > the non-constant pageblock_order case on powerpc, exclude > TRANSPARENT_HUGEPAGE and HUGETLB_PAGE from ARCH_SUPPORTS_KEXEC_HANDOVER > so the two can't be enabled together and hit this build failure. This should be a part of patch 2 I think. It is not a good idea to break the build in one patch and fix it in the next. That said, this is a debug check of sorts since SCRATCH_ALIGNMENT_BYTES is of MAX_ORDER_NR_PAGES, and pageblocks _should_ always be smaller of equal to this. Anyway, I don't think this is something you should gate by a config. I think you should either remove the static_assert(), or turn it into a runtime check. +Cc Michal. > > Cc: Aditya Gupta <adityag@linux.ibm.com> > Cc: Alexander Graf <graf@amazon.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Baoquan He <baoquan.he@linux.dev> > Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org> > Cc: Hari Bathini <hbathini@linux.ibm.com> > Cc: Madhavan Srinivasan <maddy@linux.ibm.com> > Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com> > Cc: Michael Ellerman <mpe@ellerman.id.au> > Cc: Mike Rapoport <rppt@kernel.org> > Cc: Nicholas Piggin <npiggin@gmail.com> > Cc: Pasha Tatashin <pasha.tatashin@soleen.com> > Cc: Pratyush Yadav <pratyush@kernel.org> > Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > Cc: Shivang Upadhyay <shivangu@linux.ibm.com> > Cc: Shrikanth Hegde <sshegde@linux.ibm.com> > Cc: kexec@lists.infradead.org > Cc: linux-kernel@vger.kernel.org > Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com> > --- > arch/powerpc/Kconfig | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig > index 61350d3e7a19..f68b9fc883ba 100644 > --- a/arch/powerpc/Kconfig > +++ b/arch/powerpc/Kconfig > @@ -720,6 +720,7 @@ config ARCH_SUPPORTS_KEXEC_HANDOVER > def_bool y > depends on PPC64 > depends on !CRASH_DUMP > + depends on !TRANSPARENT_HUGEPAGE && !HUGETLB_PAGE > > config ARCH_SUPPORTS_CRASH_HOTPLUG > def_bool y -- Regards, Pratyush Yadav ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE 2026-08-21 11:34 ` Pratyush Yadav @ 2026-08-23 13:52 ` Sourabh Jain 0 siblings, 0 replies; 8+ messages in thread From: Sourabh Jain @ 2026-08-23 13:52 UTC (permalink / raw) To: Pratyush Yadav Cc: linuxppc-dev, Aditya Gupta, Alexander Graf, Andrew Morton, Baoquan He, Christophe Leroy (CS GROUP), Hari Bathini, Madhavan Srinivasan, Mahesh Salgaonkar, Michael Ellerman, Mike Rapoport, Nicholas Piggin, Pasha Tatashin, Ritesh Harjani (IBM), Shivang Upadhyay, Shrikanth Hegde, kexec, linux-kernel, Michal Clapinski On 21/08/26 17:04, Pratyush Yadav wrote: > On Fri, Aug 21 2026, Sourabh Jain wrote: > >> Enabling CONFIG_KEXEC_HANDOVER together with CONFIG_TRANSPARENT_HUGEPAGE >> or CONFIG_HUGETLB_PAGE on powerpc fails to build: >> >> kernel/liveupdate/kexec_handover.c:49:39: error: expression in static assertion is not constant >> 49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES); >> ./include/linux/build_bug.h:80:56: note: in definition of macro '__static_assert' >> 80 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) >> | ^~~~ >> kernel/liveupdate/kexec_handover.c:49:1: note: in expansion of macro 'static_assert' >> 49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES); >> >> CMA_MIN_ALIGNMENT_BYTES is derived from pageblock_order, which on >> powerpc is not a compile-time constant when THP or HUGETLB_PAGE is >> enabled -- powerpc can determine huge page size at runtime in that >> configuration, so pageblock_order becomes a variable rather than a >> constant expression. KHO's static_assert() on >> SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES then fails to >> compile, since it requires both sides to be constant expressions. >> >> Until CMA_MIN_ALIGNMENT_BYTES (or KHO's use of it) is made to handle >> the non-constant pageblock_order case on powerpc, exclude >> TRANSPARENT_HUGEPAGE and HUGETLB_PAGE from ARCH_SUPPORTS_KEXEC_HANDOVER >> so the two can't be enabled together and hit this build failure. > This should be a part of patch 2 I think. It is not a good idea to break > the build in one patch and fix it in the next. Agree. Since this is an RFC patch series, I kept it separate so that this issue could be reviewed independently from patch 2. Going forward, I will merge them. > > That said, this is a debug check of sorts since SCRATCH_ALIGNMENT_BYTES > is of MAX_ORDER_NR_PAGES, and pageblocks _should_ always be smaller of > equal to this. > > Anyway, I don't think this is something you should gate by a config. I > think you should either remove the static_assert(), or turn it into a > runtime check. Agreed. I will evaluate both options and decide which approach makes more sense. Thanks for the review Pratyush. - Sourabh Jain > > +Cc Michal. > >> Cc: Aditya Gupta <adityag@linux.ibm.com> >> Cc: Alexander Graf <graf@amazon.com> >> Cc: Andrew Morton <akpm@linux-foundation.org> >> Cc: Baoquan He <baoquan.he@linux.dev> >> Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org> >> Cc: Hari Bathini <hbathini@linux.ibm.com> >> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> >> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com> >> Cc: Michael Ellerman <mpe@ellerman.id.au> >> Cc: Mike Rapoport <rppt@kernel.org> >> Cc: Nicholas Piggin <npiggin@gmail.com> >> Cc: Pasha Tatashin <pasha.tatashin@soleen.com> >> Cc: Pratyush Yadav <pratyush@kernel.org> >> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> >> Cc: Shivang Upadhyay <shivangu@linux.ibm.com> >> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> >> Cc: kexec@lists.infradead.org >> Cc: linux-kernel@vger.kernel.org >> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com> >> --- >> arch/powerpc/Kconfig | 1 + >> 1 file changed, 1 insertion(+) >> >> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig >> index 61350d3e7a19..f68b9fc883ba 100644 >> --- a/arch/powerpc/Kconfig >> +++ b/arch/powerpc/Kconfig >> @@ -720,6 +720,7 @@ config ARCH_SUPPORTS_KEXEC_HANDOVER >> def_bool y >> depends on PPC64 >> depends on !CRASH_DUMP >> + depends on !TRANSPARENT_HUGEPAGE && !HUGETLB_PAGE >> >> config ARCH_SUPPORTS_CRASH_HOTPLUG >> def_bool y ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-08-23 15:41 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-08-21 10:56 [RFC PATCH 0/3] powerpc: initial support for Kexec HandOver (KHO) Sourabh Jain 2026-08-21 10:56 ` [RFC PATCH 1/3] kernel/liveupdate: make KEXEC_HANDOVER depend on KEXEC_FILE instead of selecting it Sourabh Jain 2026-08-21 10:56 ` [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO) Sourabh Jain 2026-08-21 11:56 ` Pratyush Yadav 2026-08-23 15:41 ` Sourabh Jain 2026-08-21 10:56 ` [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE Sourabh Jain 2026-08-21 11:34 ` Pratyush Yadav 2026-08-23 13:52 ` Sourabh Jain
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox