* Re: [PATCH v2 4/5] modules: Add CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
From: Daniel Thompson @ 2022-01-28 14:09 UTC (permalink / raw)
To: Christophe Leroy
Cc: linux-arch@vger.kernel.org, kgdb-bugreport@lists.sourceforge.net,
Jason Wessel, linux-kernel@vger.kernel.org, Douglas Anderson,
linux-mm@kvack.org, Luis Chamberlain, Jessica Yu,
linuxppc-dev@lists.ozlabs.org
In-Reply-To: <af8519537d2a5c36b71a2f48ba9b81c07c93a5c4.1643282353.git.christophe.leroy@csgroup.eu>
On Thu, Jan 27, 2022 at 11:28:09AM +0000, Christophe Leroy wrote:
> Add CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC to allow architectures
> to request having modules data in vmalloc area instead of module area.
>
> This is required on powerpc book3s/32 in order to set data non
> executable, because it is not possible to set executability on page
> basis, this is done per 256 Mbytes segments. The module area has exec
> right, vmalloc area has noexec.
>
> This can also be useful on other powerpc/32 in order to maximize the
> chance of code being close enough to kernel core to avoid branch
> trampolines.
>
> Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
> Cc: Jason Wessel <jason.wessel@windriver.com>
> Cc: Daniel Thompson <daniel.thompson@linaro.org>
> Cc: Douglas Anderson <dianders@chromium.org>
Thanks for diligence in making sure kdb is up to date!
Acked-by: Daniel Thompson <daniel.thompson@linaro.org>
Daniel.
^ permalink raw reply
* Re: [PATCHv3] powerpc: mm: radix_tlb: rearrange the if-else block
From: Nathan Chancellor @ 2022-01-28 15:02 UTC (permalink / raw)
To: Anders Roxell
Cc: Arnd Bergmann, llvm, ndesaulniers, linux-kernel, linuxppc-dev
In-Reply-To: <20220128131713.299198-1-anders.roxell@linaro.org>
On Fri, Jan 28, 2022 at 02:17:13PM +0100, Anders Roxell wrote:
> Clang warns:
>
> arch/powerpc/mm/book3s64/radix_tlb.c:1191:23: error: variable 'hstart' is uninitialized when used here [-Werror,-Wuninitialized]
> __tlbiel_va_range(hstart, hend, pid,
> ^~~~~~
> arch/powerpc/mm/book3s64/radix_tlb.c:1175:23: note: initialize the variable 'hstart' to silence this warning
> unsigned long hstart, hend;
> ^
> = 0
> arch/powerpc/mm/book3s64/radix_tlb.c:1191:31: error: variable 'hend' is uninitialized when used here [-Werror,-Wuninitialized]
> __tlbiel_va_range(hstart, hend, pid,
> ^~~~
> arch/powerpc/mm/book3s64/radix_tlb.c:1175:29: note: initialize the variable 'hend' to silence this warning
> unsigned long hstart, hend;
> ^
> = 0
> 2 errors generated.
>
> Rework the 'if (IS_ENABLE(CONFIG_TRANSPARENT_HUGEPAGE))' so hstart/hend
> always gets initialized, this will silence the warnings. That will also
> simplify the 'else' path. Clang is getting confused with these warnings,
> but the warnings is a false-positive.
>
> Suggested-by: Arnd Bergmann <arnd@arndb.de>
> Suggested-by: Nathan Chancellor <nathan@kernel.org>
> Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>
> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> ---
> arch/powerpc/mm/book3s64/radix_tlb.c | 11 ++++-------
> 1 file changed, 4 insertions(+), 7 deletions(-)
>
> diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c
> index 7724af19ed7e..5172d5cec2c0 100644
> --- a/arch/powerpc/mm/book3s64/radix_tlb.c
> +++ b/arch/powerpc/mm/book3s64/radix_tlb.c
> @@ -1171,15 +1171,12 @@ static inline void __radix__flush_tlb_range(struct mm_struct *mm,
> }
> }
> } else {
> - bool hflush = false;
> + bool hflush;
> unsigned long hstart, hend;
>
> - if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) {
> - hstart = (start + PMD_SIZE - 1) & PMD_MASK;
> - hend = end & PMD_MASK;
> - if (hstart < hend)
> - hflush = true;
> - }
> + hstart = (start + PMD_SIZE - 1) & PMD_MASK;
> + hend = end & PMD_MASK;
> + hflush = IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && hstart < hend;
>
> if (type == FLUSH_TYPE_LOCAL) {
> asm volatile("ptesync": : :"memory");
> --
> 2.34.1
>
^ permalink raw reply
* [PATCH RFC v1] drivers/base/node: consolidate node device subsystem initialization in node_dev_init()
From: David Hildenbrand @ 2022-01-28 15:15 UTC (permalink / raw)
To: linux-kernel
Cc: Michal Hocko, linux-ia64, David Hildenbrand, Dave Hansen, x86,
linux-mm, Rich Felker, Paul Mackerras, sparclinux, linux-riscv,
Will Deacon, linux-s390, Yoshinori Sato, linux-sh,
Rafael J. Wysocki, Ingo Molnar, Catalin Marinas, Albert Ou,
Vasily Gorbik, Heiko Carstens, Borislav Petkov, Paul Walmsley,
Thomas Gleixner, linux-arm-kernel, Oscar Salvador,
Thomas Bogendoerfer, Greg Kroah-Hartman, linux-mips,
Palmer Dabbelt, Andrew Morton, linuxppc-dev, David S. Miller
... and call node_dev_init() after memory_dev_init() from driver_init(),
so before any of the existing arch/subsys calls. All online nodes should
be known at that point.
This is in line with memory_dev_init(), which initializes the memory
device subsystem and creates all memory block devices.
Similar to memory_dev_init(), panic() if anything goes wrong, we don't
want to continue with such basic initialization errors.
The important part is that node_dev_init() gets called after
memory_dev_init() and after cpu_dev_init(), but before any of the
relevant archs call register_cpu() to register the new cpu device under
the node device. The latter should be the case for the current users
of topology_init().
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: x86@kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-mips@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-riscv@lists.infradead.org
Cc: linux-s390@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-mm@kvack.org
Signed-off-by: David Hildenbrand <david@redhat.com>
---
RFC because I tested only on x86-64 and s390x, I think I cross-compiled all
applicable architectures except riscv and sparc.
This is somewhat a preparation for detecting if a memory block
(/sys/devices/system/memory/memory*) is managed by a single zone, and
storing the zone for the memory block -- to get rid of
test_pages_in_a_zone(). For that, we want to know all nodes that are
applicable for a single memory block (mem->nid), which is determined when
registering the node.
While this change might not be strictly required for that change, this
way it's easier to see when the nodes are gettin created and
consequently when the node ids for a memory block are determined.
---
arch/arm64/kernel/setup.c | 3 ---
arch/ia64/kernel/topology.c | 10 ----------
arch/mips/kernel/topology.c | 5 -----
arch/powerpc/kernel/sysfs.c | 17 -----------------
arch/riscv/kernel/setup.c | 3 ---
arch/s390/kernel/numa.c | 7 -------
arch/sh/kernel/topology.c | 5 -----
arch/sparc/kernel/sysfs.c | 12 ------------
arch/x86/kernel/topology.c | 5 -----
drivers/base/init.c | 1 +
drivers/base/node.c | 30 +++++++++++++++++-------------
include/linux/node.h | 4 ++++
12 files changed, 22 insertions(+), 80 deletions(-)
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index f70573928f1b..3505789cf4bd 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -406,9 +406,6 @@ static int __init topology_init(void)
{
int i;
- for_each_online_node(i)
- register_one_node(i);
-
for_each_possible_cpu(i) {
struct cpu *cpu = &per_cpu(cpu_data.cpu, i);
cpu->hotpluggable = cpu_can_disable(i);
diff --git a/arch/ia64/kernel/topology.c b/arch/ia64/kernel/topology.c
index e4992917a24b..94a848b06f15 100644
--- a/arch/ia64/kernel/topology.c
+++ b/arch/ia64/kernel/topology.c
@@ -70,16 +70,6 @@ static int __init topology_init(void)
{
int i, err = 0;
-#ifdef CONFIG_NUMA
- /*
- * MCD - Do we want to register all ONLINE nodes, or all POSSIBLE nodes?
- */
- for_each_online_node(i) {
- if ((err = register_one_node(i)))
- goto out;
- }
-#endif
-
sysfs_cpus = kcalloc(NR_CPUS, sizeof(struct ia64_cpu), GFP_KERNEL);
if (!sysfs_cpus)
panic("kzalloc in topology_init failed - NR_CPUS too big?");
diff --git a/arch/mips/kernel/topology.c b/arch/mips/kernel/topology.c
index 08ad6371fbe0..9429d85a4703 100644
--- a/arch/mips/kernel/topology.c
+++ b/arch/mips/kernel/topology.c
@@ -12,11 +12,6 @@ static int __init topology_init(void)
{
int i, ret;
-#ifdef CONFIG_NUMA
- for_each_online_node(i)
- register_one_node(i);
-#endif /* CONFIG_NUMA */
-
for_each_present_cpu(i) {
struct cpu *c = &per_cpu(cpu_devices, i);
diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
index d45a415d5374..2069bbb90a9a 100644
--- a/arch/powerpc/kernel/sysfs.c
+++ b/arch/powerpc/kernel/sysfs.c
@@ -1110,14 +1110,6 @@ EXPORT_SYMBOL_GPL(cpu_remove_dev_attr_group);
/* NUMA stuff */
#ifdef CONFIG_NUMA
-static void __init register_nodes(void)
-{
- int i;
-
- for (i = 0; i < MAX_NUMNODES; i++)
- register_one_node(i);
-}
-
int sysfs_add_device_to_node(struct device *dev, int nid)
{
struct node *node = node_devices[nid];
@@ -1132,13 +1124,6 @@ void sysfs_remove_device_from_node(struct device *dev, int nid)
sysfs_remove_link(&node->dev.kobj, kobject_name(&dev->kobj));
}
EXPORT_SYMBOL_GPL(sysfs_remove_device_from_node);
-
-#else
-static void __init register_nodes(void)
-{
- return;
-}
-
#endif
/* Only valid if CPU is present. */
@@ -1155,8 +1140,6 @@ static int __init topology_init(void)
{
int cpu, r;
- register_nodes();
-
for_each_possible_cpu(cpu) {
struct cpu *c = &per_cpu(cpu_devices, cpu);
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index b42bfdc67482..834eb652a7b9 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -301,9 +301,6 @@ static int __init topology_init(void)
{
int i, ret;
- for_each_online_node(i)
- register_one_node(i);
-
for_each_possible_cpu(i) {
struct cpu *cpu = &per_cpu(cpu_devices, i);
diff --git a/arch/s390/kernel/numa.c b/arch/s390/kernel/numa.c
index 51c5a9f6e525..23ab9f02f278 100644
--- a/arch/s390/kernel/numa.c
+++ b/arch/s390/kernel/numa.c
@@ -33,10 +33,3 @@ void __init numa_setup(void)
NODE_DATA(0)->node_spanned_pages = memblock_end_of_DRAM() >> PAGE_SHIFT;
NODE_DATA(0)->node_id = 0;
}
-
-static int __init numa_init_late(void)
-{
- register_one_node(0);
- return 0;
-}
-arch_initcall(numa_init_late);
diff --git a/arch/sh/kernel/topology.c b/arch/sh/kernel/topology.c
index 76af6db9daa2..2d2a7509b565 100644
--- a/arch/sh/kernel/topology.c
+++ b/arch/sh/kernel/topology.c
@@ -46,11 +46,6 @@ static int __init topology_init(void)
{
int i, ret;
-#ifdef CONFIG_NUMA
- for_each_online_node(i)
- register_one_node(i);
-#endif
-
for_each_present_cpu(i) {
struct cpu *c = &per_cpu(cpu_devices, i);
diff --git a/arch/sparc/kernel/sysfs.c b/arch/sparc/kernel/sysfs.c
index 6d60d416f0dd..f19487e4cc71 100644
--- a/arch/sparc/kernel/sysfs.c
+++ b/arch/sparc/kernel/sysfs.c
@@ -244,22 +244,10 @@ static void __init check_mmu_stats(void)
mmu_stats_supported = 1;
}
-static void register_nodes(void)
-{
-#ifdef CONFIG_NUMA
- int i;
-
- for (i = 0; i < MAX_NUMNODES; i++)
- register_one_node(i);
-#endif
-}
-
static int __init topology_init(void)
{
int cpu, ret;
- register_nodes();
-
check_mmu_stats();
for_each_possible_cpu(cpu) {
diff --git a/arch/x86/kernel/topology.c b/arch/x86/kernel/topology.c
index bd83748e2bde..8617d1ed9d31 100644
--- a/arch/x86/kernel/topology.c
+++ b/arch/x86/kernel/topology.c
@@ -154,11 +154,6 @@ static int __init topology_init(void)
{
int i;
-#ifdef CONFIG_NUMA
- for_each_online_node(i)
- register_one_node(i);
-#endif
-
for_each_present_cpu(i)
arch_register_cpu(i);
diff --git a/drivers/base/init.c b/drivers/base/init.c
index a9f57c22fb9e..d8d0fe687111 100644
--- a/drivers/base/init.c
+++ b/drivers/base/init.c
@@ -35,5 +35,6 @@ void __init driver_init(void)
auxiliary_bus_init();
cpu_dev_init();
memory_dev_init();
+ node_dev_init();
container_dev_init();
}
diff --git a/drivers/base/node.c b/drivers/base/node.c
index 87acc47e8951..a133981a12fc 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -1065,26 +1065,30 @@ static const struct attribute_group *cpu_root_attr_groups[] = {
};
#define NODE_CALLBACK_PRI 2 /* lower than SLAB */
-static int __init register_node_type(void)
+void __init node_dev_init(void)
{
- int ret;
+ static struct notifier_block node_memory_callback_nb = {
+ .notifier_call = node_memory_callback,
+ .priority = NODE_CALLBACK_PRI,
+ };
+ int ret, i;
BUILD_BUG_ON(ARRAY_SIZE(node_state_attr) != NR_NODE_STATES);
BUILD_BUG_ON(ARRAY_SIZE(node_state_attrs)-1 != NR_NODE_STATES);
ret = subsys_system_register(&node_subsys, cpu_root_attr_groups);
- if (!ret) {
- static struct notifier_block node_memory_callback_nb = {
- .notifier_call = node_memory_callback,
- .priority = NODE_CALLBACK_PRI,
- };
- register_hotmemory_notifier(&node_memory_callback_nb);
- }
+ if (ret)
+ panic("%s() failed to register subsystem: %d\n", __func__, ret);
+
+ register_hotmemory_notifier(&node_memory_callback_nb);
/*
- * Note: we're not going to unregister the node class if we fail
- * to register the node state class attribute files.
+ * Create all node devices, which will properly link the node
+ * to applicable memory block devices and already created cpu devices.
*/
- return ret;
+ for_each_online_node(i) {
+ ret = register_one_node(i);
+ if (ret)
+ panic("%s() failed to add node: %d\n", __func__, ret);
+ }
}
-postcore_initcall(register_node_type);
diff --git a/include/linux/node.h b/include/linux/node.h
index bb21fd631b16..f3be6ccfebed 100644
--- a/include/linux/node.h
+++ b/include/linux/node.h
@@ -112,6 +112,7 @@ static inline void link_mem_sections(int nid, unsigned long start_pfn,
extern void unregister_node(struct node *node);
#ifdef CONFIG_NUMA
+extern void node_dev_init(void);
/* Core of the node registration - only memory hotplug should use this */
extern int __register_one_node(int nid);
@@ -149,6 +150,9 @@ extern void register_hugetlbfs_with_node(node_registration_func_t doregister,
node_registration_func_t unregister);
#endif
#else
+static inline void node_dev_init(void)
+{
+}
static inline int __register_one_node(int nid)
{
return 0;
base-commit: e783362eb54cd99b2cac8b3a9aeac942e6f6ac07
--
2.34.1
^ permalink raw reply related
* Re: [PATCH v3] PCI hotplug: rpaphp: Error out on busy status from get-sensor-state
From: Mahesh J Salgaonkar @ 2022-01-28 15:06 UTC (permalink / raw)
To: Nathan Lynch; +Cc: Tyrel Datwyler, Oliver O'Halloran, linuxppc-dev
In-Reply-To: <87y24tn7ms.fsf@linux.ibm.com>
On 2021-12-09 09:02:51 Thu, Nathan Lynch wrote:
> Mahesh Salgaonkar <mahesh@linux.ibm.com> writes:
> > To avoid this issue, fix the pci hotplug driver (rpaphp) to return an error
> > if the slot presence state can not be detected immediately. Current
> > implementation uses rtas_get_sensor() API which blocks the slot check state
> > until rtas call returns success. Change rpaphp_get_sensor_state() to invoke
> > rtas_call(get-sensor-state) directly and take actions based on rtas return
> > status. This patch now errors out immediately on busy return status from
> > rtas_call.
> >
> > Please note that, only on certain PHB failures, the slot presence check
> > returns BUSY condition. In normal cases it returns immediately with a
> > correct presence state value. Hence this change has no impact on normal pci
> > dlpar operations.
>
> I was wondering about this. This seems to be saying -2/990x cannot
> happen in other cases. I couldn't find this specified in the
> architecture. It seems a bit risky to me to *always* error out on
> -2/990x - won't we have intermittent slot enable failures?
Sorry for the late response. So instead of always returning error out
how about we error out only if pe is going through EEH recovery ? During
get_adapter_status I can check if pe->state is set to EEH_PE_RECOVERING
and only then return error on busy else fallback to existing method of
rtas_get_sensor. Let me send out another version with this approach.
Thanks,
-Mahesh.
>
> > +/*
> > + * RTAS call get-sensor-state(DR_ENTITY_SENSE) return values as per PAPR:
> > + * -1: Hardware Error
> > + * -2: RTAS_BUSY
> > + * -3: Invalid sensor. RTAS Parameter Error.
> > + * -9000: Need DR entity to be powered up and unisolated before RTAS call
> > + * -9001: Need DR entity to be powered up, but not unisolated, before RTAS call
> > + * -9002: DR entity unusable
> > + * 990x: Extended delay - where x is a number in the range of 0-5
> > + */
> > +#define RTAS_HARDWARE_ERROR -1
> > +#define RTAS_INVALID_SENSOR -3
> > +#define SLOT_UNISOLATED -9000
> > +#define SLOT_NOT_UNISOLATED -9001
> > +#define SLOT_NOT_USABLE -9002
> > +
> > +static int rtas_to_errno(int rtas_rc)
> > +{
> > + int rc;
> > +
> > + switch (rtas_rc) {
> > + case RTAS_HARDWARE_ERROR:
> > + rc = -EIO;
> > + break;
> > + case RTAS_INVALID_SENSOR:
> > + rc = -EINVAL;
> > + break;
> > + case SLOT_UNISOLATED:
> > + case SLOT_NOT_UNISOLATED:
> > + rc = -EFAULT;
> > + break;
> > + case SLOT_NOT_USABLE:
> > + rc = -ENODEV;
> > + break;
> > + case RTAS_BUSY:
> > + case RTAS_EXTENDED_DELAY_MIN...RTAS_EXTENDED_DELAY_MAX:
> > + rc = -EBUSY;
> > + break;
> > + default:
> > + err("%s: unexpected RTAS error %d\n", __func__, rtas_rc);
> > + rc = -ERANGE;
> > + break;
> > + }
> > + return rc;
> > +}
>
> These conversions look OK to me.
--
Mahesh J Salgaonkar
^ permalink raw reply
* Re: ftrace hangs waiting for rcu (was: Re: [PATCH] ftrace: Have architectures opt-in for mcount build time sorting)
From: Mark Rutland @ 2022-01-28 15:42 UTC (permalink / raw)
To: Sven Schnelle
Cc: Kees Cook, Paul E. McKenney, hca, LKML, Steven Rostedt,
Ingo Molnar, linux-s390, Sachin Sant, Russell King, Andrew Morton,
Yinan Liu, linuxppc-dev, Ard Biesheuvel, linux-arm-kernel
In-Reply-To: <yt9dy231yq90.fsf_-_@linux.ibm.com>
Hi Sven,
On Thu, Jan 27, 2022 at 07:42:35PM +0100, Sven Schnelle wrote:
> Mark Rutland <mark.rutland@arm.com> writes:
>
> > * I intermittently see a hang when running the tests. I previously hit that
> > when originally trying to bisect this issue (and IIRC that bisected down to
> > some RCU changes, but I need to re-run that). When the tests hang I
> > magic-srsrq + L tells me:
> >
> > [ 271.938438] sysrq: Show Blocked State
> > [ 271.939245] task:ftracetest state:D stack: 0 pid: 5687 ppid: 5627 flags:0x00000200
> > [ 271.940961] Call trace:
> > [ 271.941472] __switch_to+0x104/0x160
> > [ 271.942213] __schedule+0x2b0/0x6e0
> > [ 271.942933] schedule+0x5c/0xf0
> > [ 271.943586] schedule_timeout+0x184/0x1c4
> > [ 271.944410] wait_for_completion+0x8c/0x12c
> > [ 271.945274] __wait_rcu_gp+0x184/0x190
> > [ 271.946047] synchronize_rcu_tasks_rude+0x48/0x70
> > [ 271.947007] update_ftrace_function+0xa4/0xec
> > [ 271.947897] __unregister_ftrace_function+0xa4/0xf0
> > [ 271.948898] unregister_ftrace_function+0x34/0x70
> > [ 271.949857] wakeup_tracer_reset+0x4c/0x100
> > [ 271.950713] tracing_set_tracer+0xd0/0x2b0
> > [ 271.951552] tracing_set_trace_write+0xe8/0x150
> > [ 271.952477] vfs_write+0xfc/0x284
> > [ 271.953171] ksys_write+0x7c/0x110
> > [ 271.953874] __arm64_sys_write+0x2c/0x40
> > [ 271.954678] invoke_syscall+0x5c/0x130
> > [ 271.955442] el0_svc_common.constprop.0+0x108/0x130
> > [ 271.956435] do_el0_svc+0x74/0x90
> > [ 271.957124] el0_svc+0x2c/0x90
> > [ 271.957757] el0t_64_sync_handler+0xa8/0x12c
> > [ 271.958629] el0t_64_sync+0x1a0/0x1a4
On arm64 I bisected this down to:
7a30871b6a27de1a ("rcu-tasks: Introduce ->percpu_enqueue_shift for dynamic queue selection")
Which was going wrong because ilog2() rounds down, and so the shift was wrong
for any nr_cpus that was not a power-of-two. Paul had already fixed that in
rcu-next, and just sent a pull request to Linus:
https://lore.kernel.org/lkml/20220128143251.GA2398275@paulmck-ThinkPad-P17-Gen-1/
With that applied, I no longer see these hangs.
Does your s390 test machine have a non-power-of-two nr_cpus, and does that fix
the issue for you?
On arm64 the startup tests didn't seem to trigger the hang, but I was able to
trigger the hang fairly reliably with the ftrace selftests, e.g.
$ for N in $(seq 1 10); do ./ftracetest test.d/00basic/basic2.tc; done
... which prior to the fix, would hang between runs 2 to 5.
Thanks,
Mark.
> that's interesting. On s390 i'm seeing the same problem in CI, but with
> the startup ftrace tests. So that's likely not arm64 spacific.
>
> On s390, the last messages from ftrace are [ 5.663568] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> [ 5.667099] futex hash table entries: 65536 (order: 12, 16777216 bytes, vmalloc)
> [ 5.739549] Running postponed tracer tests:
> [ 5.740662] Testing tracer function: PASSED
> [ 6.194635] Testing dynamic ftrace: PASSED
> [ 6.471213] Testing dynamic ftrace ops #1:
> [ 6.558445] (1 0 1 0 0)
> [ 6.558458] (1 1 2 0 0)
> [ 6.699135] (2 1 3 0 764347)
> [ 6.699252] (2 2 4 0 766466)
> [ 6.759857] (3 2 4 0 1159604)
> [..] hangs here
>
> The backtrace looks like this, which is very similar to the one above:
>
> crash> bt 1
> PID: 1 TASK: 80e68100 CPU: 133 COMMAND: "swapper/0"
> #0 [380004df808] __schedule at cda39f0e
> #1 [380004df880] schedule at cda3a488
> #2 [380004df8b0] schedule_timeout at cda41ef6
> #3 [380004df978] wait_for_completion at cda3bd0a
> #4 [380004df9d8] __wait_rcu_gp at ccdddd92
> #5 [380004dfa30] synchronize_rcu_tasks_generic at ccdde0aa
> #6 [380004dfad8] ftrace_shutdown at cce7b050
> #7 [380004dfb18] unregister_ftrace_function at cce7b192
> #8 [380004dfb50] trace_selftest_ops at cda1e0fa
> #9 [380004dfba0] run_tracer_selftest at cda1e4f2
> #10 [380004dfc00] trace_selftest_startup_function at ce74355c
> #11 [380004dfc58] run_tracer_selftest at cda1e2fc
> #12 [380004dfc98] init_trace_selftests at ce742d30
> #13 [380004dfcd0] do_one_initcall at cccdca16
> #14 [380004dfd68] do_initcalls at ce72e776
> #15 [380004dfde0] kernel_init_freeable at ce72ea60
> #16 [380004dfe50] kernel_init at cda333fe
> #17 [380004dfe68] __ret_from_fork at cccdf920
> #18 [380004dfe98] ret_from_fork at cda444ca
>
> I didn't had success reproducing it so far, but it is good to know that
> this also happens when running the ftrace testsuite.
>
> I have several crashdumps, so i could try to pull out some information
> if someone tells me what to look for.
>
> Thanks,
> Sven
^ permalink raw reply
* Re: ftrace hangs waiting for rcu
From: Sven Schnelle @ 2022-01-28 16:08 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-s390, Kees Cook, Paul E. McKenney, hca, LKML,
Steven Rostedt, Ingo Molnar, Sachin Sant, Russell King,
Andrew Morton, Yinan Liu, linuxppc-dev, Ard Biesheuvel,
linux-arm-kernel
In-Reply-To: <YfQCohKWJg9H+uID@FVFF77S0Q05N>
Hi Mark,
Mark Rutland <mark.rutland@arm.com> writes:
> On arm64 I bisected this down to:
>
> 7a30871b6a27de1a ("rcu-tasks: Introduce ->percpu_enqueue_shift for dynamic queue selection")
>
> Which was going wrong because ilog2() rounds down, and so the shift was wrong
> for any nr_cpus that was not a power-of-two. Paul had already fixed that in
> rcu-next, and just sent a pull request to Linus:
>
> https://lore.kernel.org/lkml/20220128143251.GA2398275@paulmck-ThinkPad-P17-Gen-1/
>
> With that applied, I no longer see these hangs.
>
> Does your s390 test machine have a non-power-of-two nr_cpus, and does that fix
> the issue for you?
We noticed the PR from Paul and are currently testing the fix. So far
it's looking good. The configuration where we have seen the hang is a
bit unusual:
- 16 physical CPUs on the kvm host
- 248 logical CPUs inside kvm
- debug kernel both on the host and kvm guest
So things are likely a bit slow in the kvm guest. Interesting is that
the number of CPUs is even. But maybe RCU sees an odd number of CPUs
and gets confused before all cpus are brought up. Have to read code/test
to see whether that could be possible.
Thanks for investigating!
Sven
^ permalink raw reply
* Re: ftrace hangs waiting for rcu
From: Mark Rutland @ 2022-01-28 16:11 UTC (permalink / raw)
To: Sven Schnelle
Cc: linux-s390, Kees Cook, Paul E. McKenney, hca, LKML,
Steven Rostedt, Ingo Molnar, Sachin Sant, Russell King,
Andrew Morton, Yinan Liu, linuxppc-dev, Ard Biesheuvel,
linux-arm-kernel
In-Reply-To: <yt9dee4rn8q7.fsf@linux.ibm.com>
On Fri, Jan 28, 2022 at 05:08:48PM +0100, Sven Schnelle wrote:
> Hi Mark,
>
> Mark Rutland <mark.rutland@arm.com> writes:
>
> > On arm64 I bisected this down to:
> >
> > 7a30871b6a27de1a ("rcu-tasks: Introduce ->percpu_enqueue_shift for dynamic queue selection")
> >
> > Which was going wrong because ilog2() rounds down, and so the shift was wrong
> > for any nr_cpus that was not a power-of-two. Paul had already fixed that in
> > rcu-next, and just sent a pull request to Linus:
> >
> > https://lore.kernel.org/lkml/20220128143251.GA2398275@paulmck-ThinkPad-P17-Gen-1/
> >
> > With that applied, I no longer see these hangs.
> >
> > Does your s390 test machine have a non-power-of-two nr_cpus, and does that fix
> > the issue for you?
>
> We noticed the PR from Paul and are currently testing the fix. So far
> it's looking good. The configuration where we have seen the hang is a
> bit unusual:
>
> - 16 physical CPUs on the kvm host
> - 248 logical CPUs inside kvm
Aha! 248 is notably *NOT* a power of two, and in this case the shift would be
wrong (ilog2() would give 7, when we need a shift of 8).
So I suspect you're hitting the same issue as I was.
Thanks,
Mark.
> - debug kernel both on the host and kvm guest
>
> So things are likely a bit slow in the kvm guest. Interesting is that
> the number of CPUs is even. But maybe RCU sees an odd number of CPUs
> and gets confused before all cpus are brought up. Have to read code/test
> to see whether that could be possible.
>
> Thanks for investigating!
> Sven
^ permalink raw reply
* Re: ftrace hangs waiting for rcu
From: Paul E. McKenney @ 2022-01-28 16:15 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-s390, Kees Cook, hca, LKML, Steven Rostedt, Ard Biesheuvel,
Sven Schnelle, Sachin Sant, Russell King, Andrew Morton,
Yinan Liu, linuxppc-dev, Ingo Molnar, linux-arm-kernel
In-Reply-To: <YfQVzba5thVs+qap@FVFF77S0Q05N>
On Fri, Jan 28, 2022 at 04:11:57PM +0000, Mark Rutland wrote:
> On Fri, Jan 28, 2022 at 05:08:48PM +0100, Sven Schnelle wrote:
> > Hi Mark,
> >
> > Mark Rutland <mark.rutland@arm.com> writes:
> >
> > > On arm64 I bisected this down to:
> > >
> > > 7a30871b6a27de1a ("rcu-tasks: Introduce ->percpu_enqueue_shift for dynamic queue selection")
> > >
> > > Which was going wrong because ilog2() rounds down, and so the shift was wrong
> > > for any nr_cpus that was not a power-of-two. Paul had already fixed that in
> > > rcu-next, and just sent a pull request to Linus:
> > >
> > > https://lore.kernel.org/lkml/20220128143251.GA2398275@paulmck-ThinkPad-P17-Gen-1/
> > >
> > > With that applied, I no longer see these hangs.
> > >
> > > Does your s390 test machine have a non-power-of-two nr_cpus, and does that fix
> > > the issue for you?
> >
> > We noticed the PR from Paul and are currently testing the fix. So far
> > it's looking good. The configuration where we have seen the hang is a
> > bit unusual:
> >
> > - 16 physical CPUs on the kvm host
> > - 248 logical CPUs inside kvm
>
> Aha! 248 is notably *NOT* a power of two, and in this case the shift would be
> wrong (ilog2() would give 7, when we need a shift of 8).
>
> So I suspect you're hitting the same issue as I was.
And apparently no one runs -next on systems having a non-power-of-two
number of CPUs. ;-)
Thanx, Paul
> Thanks,
> Mark.
>
> > - debug kernel both on the host and kvm guest
> >
> > So things are likely a bit slow in the kvm guest. Interesting is that
> > the number of CPUs is even. But maybe RCU sees an odd number of CPUs
> > and gets confused before all cpus are brought up. Have to read code/test
> > to see whether that could be possible.
> >
> > Thanks for investigating!
> > Sven
^ permalink raw reply
* Re: ftrace hangs waiting for rcu
From: Sven Schnelle @ 2022-01-28 16:17 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-s390, Kees Cook, Paul E. McKenney, hca, LKML,
Steven Rostedt, Ingo Molnar, Sachin Sant, Russell King,
Andrew Morton, Yinan Liu, linuxppc-dev, Ard Biesheuvel,
linux-arm-kernel
In-Reply-To: <YfQVzba5thVs+qap@FVFF77S0Q05N>
Hi Mark,
Mark Rutland <mark.rutland@arm.com> writes:
> On Fri, Jan 28, 2022 at 05:08:48PM +0100, Sven Schnelle wrote:
>> We noticed the PR from Paul and are currently testing the fix. So far
>> it's looking good. The configuration where we have seen the hang is a
>> bit unusual:
>>
>> - 16 physical CPUs on the kvm host
>> - 248 logical CPUs inside kvm
>
> Aha! 248 is notably *NOT* a power of two, and in this case the shift would be
> wrong (ilog2() would give 7, when we need a shift of 8).
>
> So I suspect you're hitting the same issue as I was.
Argh, indeed! I somehow changed 'power of two' to 'odd number' in my
head. I guess it's time for the weekend. :-)
Thanks!
^ permalink raw reply
* Re: ftrace hangs waiting for rcu
From: Paul E. McKenney @ 2022-01-28 17:47 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-s390, Kees Cook, andriin, hca, LKML, Steven Rostedt,
Ard Biesheuvel, Sven Schnelle, Sachin Sant, Russell King,
Andrew Morton, Yinan Liu, linuxppc-dev, Ingo Molnar,
linux-arm-kernel
In-Reply-To: <20220128161547.GL4285@paulmck-ThinkPad-P17-Gen-1>
On Fri, Jan 28, 2022 at 08:15:47AM -0800, Paul E. McKenney wrote:
> On Fri, Jan 28, 2022 at 04:11:57PM +0000, Mark Rutland wrote:
> > On Fri, Jan 28, 2022 at 05:08:48PM +0100, Sven Schnelle wrote:
> > > Hi Mark,
> > >
> > > Mark Rutland <mark.rutland@arm.com> writes:
> > >
> > > > On arm64 I bisected this down to:
> > > >
> > > > 7a30871b6a27de1a ("rcu-tasks: Introduce ->percpu_enqueue_shift for dynamic queue selection")
> > > >
> > > > Which was going wrong because ilog2() rounds down, and so the shift was wrong
> > > > for any nr_cpus that was not a power-of-two. Paul had already fixed that in
> > > > rcu-next, and just sent a pull request to Linus:
> > > >
> > > > https://lore.kernel.org/lkml/20220128143251.GA2398275@paulmck-ThinkPad-P17-Gen-1/
> > > >
> > > > With that applied, I no longer see these hangs.
> > > >
> > > > Does your s390 test machine have a non-power-of-two nr_cpus, and does that fix
> > > > the issue for you?
> > >
> > > We noticed the PR from Paul and are currently testing the fix. So far
> > > it's looking good. The configuration where we have seen the hang is a
> > > bit unusual:
> > >
> > > - 16 physical CPUs on the kvm host
> > > - 248 logical CPUs inside kvm
> >
> > Aha! 248 is notably *NOT* a power of two, and in this case the shift would be
> > wrong (ilog2() would give 7, when we need a shift of 8).
> >
> > So I suspect you're hitting the same issue as I was.
>
> And apparently no one runs -next on systems having a non-power-of-two
> number of CPUs. ;-)
And the fix is now in mainline.
Thanx, Paul
> > Thanks,
> > Mark.
> >
> > > - debug kernel both on the host and kvm guest
> > >
> > > So things are likely a bit slow in the kvm guest. Interesting is that
> > > the number of CPUs is even. But maybe RCU sees an odd number of CPUs
> > > and gets confused before all cpus are brought up. Have to read code/test
> > > to see whether that could be possible.
> > >
> > > Thanks for investigating!
> > > Sven
^ permalink raw reply
* Re: [PATCH] ftrace: Have architectures opt-in for mcount build time sorting
From: Joe Lawrence @ 2022-01-28 21:11 UTC (permalink / raw)
To: Steven Rostedt
Cc: Mark Rutland, Kees Cook, LKML, Ingo Molnar, Sachin Sant,
Russell King, Andrew Morton, Yinan Liu, linuxppc-dev,
Ard Biesheuvel, linux-arm-kernel
In-Reply-To: <20220127114249.03b1b52b@gandalf.local.home>
On Thu, Jan 27, 2022 at 11:42:49AM -0500, Steven Rostedt wrote:
> From: "Steven Rostedt (Google)" <rostedt@goodmis.org>
>
> First S390 complained that the sorting of the mcount sections at build
> time caused the kernel to crash on their architecture. Now PowerPC is
> complaining about it too. And also ARM64 appears to be having issues.
>
> It may be necessary to also update the relocation table for the values
> in the mcount table. Not only do we have to sort the table, but also
> update the relocations that may be applied to the items in the table.
>
> If the system is not relocatable, then it is fine to sort, but if it is,
> some architectures may have issues (although x86 does not as it shifts all
> addresses the same).
>
> Add a HAVE_BUILDTIME_MCOUNT_SORT that an architecture can set to say it is
> safe to do the sorting at build time.
>
> Also update the config to compile in build time sorting in the sorttable
> code in scripts/ to depend on CONFIG_BUILDTIME_MCOUNT_SORT.
>
> Link: https://lore.kernel.org/all/944D10DA-8200-4BA9-8D0A-3BED9AA99F82@linux.ibm.com/
>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Yinan Liu <yinan@linux.alibaba.com>
> Cc: Ard Biesheuvel <ardb@kernel.org>
> Cc: Kees Cook <keescook@chromium.org>
> Cc: linuxppc-dev@lists.ozlabs.org
> Reported-by: Sachin Sant <sachinp@linux.ibm.com>
> Tested-by: Sachin Sant <sachinp@linux.ibm.com>
> Fixes: 72b3942a173c ("scripts: ftrace - move the sort-processing in ftrace_init")
> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> ---
> arch/arm/Kconfig | 1 +
> arch/x86/Kconfig | 1 +
> kernel/trace/Kconfig | 8 +++++++-
> scripts/Makefile | 2 +-
> 4 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index c2724d986fa0..5256ebe57451 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -82,6 +82,7 @@ config ARM
> select HAVE_EBPF_JIT if !CPU_ENDIAN_BE32
> select HAVE_CONTEXT_TRACKING
> select HAVE_C_RECORDMCOUNT
> + select HAVE_BUILDTIME_MCOUNT_SORT
> select HAVE_DEBUG_KMEMLEAK if !XIP_KERNEL
> select HAVE_DMA_CONTIGUOUS if MMU
> select HAVE_DYNAMIC_FTRACE if !XIP_KERNEL && !CPU_ENDIAN_BE32 && MMU
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 7399327d1eff..46080dea5dba 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -186,6 +186,7 @@ config X86
> select HAVE_CONTEXT_TRACKING_OFFSTACK if HAVE_CONTEXT_TRACKING
> select HAVE_C_RECORDMCOUNT
> select HAVE_OBJTOOL_MCOUNT if STACK_VALIDATION
> + select HAVE_BUILDTIME_MCOUNT_SORT
> select HAVE_DEBUG_KMEMLEAK
> select HAVE_DMA_CONTIGUOUS
> select HAVE_DYNAMIC_FTRACE
> diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
> index 752ed89a293b..7e5b92090faa 100644
> --- a/kernel/trace/Kconfig
> +++ b/kernel/trace/Kconfig
> @@ -70,10 +70,16 @@ config HAVE_C_RECORDMCOUNT
> help
> C version of recordmcount available?
>
> +config HAVE_BUILDTIME_MCOUNT_SORT
> + bool
> + help
> + An architecture selects this if it sorts the mcount_loc section
> + at build time.
> +
> config BUILDTIME_MCOUNT_SORT
> bool
> default y
> - depends on BUILDTIME_TABLE_SORT && !S390
> + depends on HAVE_BUILDTIME_MCOUNT_SORT
> help
> Sort the mcount_loc section at build time.
>
> diff --git a/scripts/Makefile b/scripts/Makefile
> index b082d2f93357..cedc1f0e21d8 100644
> --- a/scripts/Makefile
> +++ b/scripts/Makefile
> @@ -32,7 +32,7 @@ HOSTCFLAGS_sorttable.o += -I$(srctree)/tools/arch/x86/include
> HOSTCFLAGS_sorttable.o += -DUNWINDER_ORC_ENABLED
> endif
>
> -ifdef CONFIG_DYNAMIC_FTRACE
> +ifdef CONFIG_BUILDTIME_MCOUNT_SORT
> HOSTCFLAGS_sorttable.o += -DMCOUNT_SORT_ENABLED
> endif
>
> --
> 2.33.0
>
Hi Steve,
I just finished bisecting what is probably the same problem... when
running the livepatch selftests for 5.17-rc1, x86_64 passes, but I kept
getting errors like this on ppc64le:
kernel: livepatch: enabling patch 'test_klp_livepatch'
kernel: livepatch: failed to find location for function 'cmdline_proc_show'
kernel: livepatch: failed to patch object 'vmlinux'
kernel: livepatch: failed to enable patch 'test_klp_livepatch'
kernel: livepatch: 'test_klp_livepatch': unpatching complete
which means klp_get_ftrace_location() / ftrace_location_range() hit a
problem with that function.
The bisect finally landed on:
72b3942a173c387b27860ba1069636726e208777 is the first bad commit
commit 72b3942a173c387b27860ba1069636726e208777
Author: Yinan Liu <yinan@linux.alibaba.com>
Date: Sun Dec 12 19:33:58 2021 +0800
scripts: ftrace - move the sort-processing in ftrace_init
and I can confirm that your updates today in "[for-linus][PATCH 00/10]
tracing: Fixes for 5.17-rc1" fix or avoid the issue. I just wanted to
add my report in case this adds any future complications for mcount
build time sorting. Let me know if any additional tests would be
helpful.
Regards,
-- Joe
^ permalink raw reply
* Re: [PATCH] ftrace: Have architectures opt-in for mcount build time sorting
From: Steven Rostedt @ 2022-01-28 21:39 UTC (permalink / raw)
To: Joe Lawrence
Cc: Mark Rutland, Kees Cook, LKML, Ingo Molnar, Sachin Sant,
Russell King, Andrew Morton, Yinan Liu, linuxppc-dev,
Ard Biesheuvel, linux-arm-kernel
In-Reply-To: <YfRcC/ggq90jw7Uc@redhat.com>
On Fri, 28 Jan 2022 16:11:39 -0500
Joe Lawrence <joe.lawrence@redhat.com> wrote:
> The bisect finally landed on:
>
> 72b3942a173c387b27860ba1069636726e208777 is the first bad commit
> commit 72b3942a173c387b27860ba1069636726e208777
> Author: Yinan Liu <yinan@linux.alibaba.com>
> Date: Sun Dec 12 19:33:58 2021 +0800
>
> scripts: ftrace - move the sort-processing in ftrace_init
>
> and I can confirm that your updates today in "[for-linus][PATCH 00/10]
> tracing: Fixes for 5.17-rc1" fix or avoid the issue. I just wanted to
> add my report in case this adds any future complications for mcount
> build time sorting. Let me know if any additional tests would be
> helpful.
Thanks for letting me know. That patch set has already landed in Linus's
tree.
-- Steve
^ permalink raw reply
* [PATCH 0/3] powerpc/pseries/vas: VAS/NXGZIP support with LPM
From: Haren Myneni @ 2022-01-29 1:18 UTC (permalink / raw)
To: mpe, linuxppc-dev, npiggin, nathanl; +Cc: haren
Virtual Accelerator Switchboard (VAS) is an engine stays on the
chip. So all windows opened on a specific engine belongs to VAS
the chip. The hypervisor expects the partition to close all
active windows on the sources system and reopen them after
migration on the destination machine.
This patch series adds VAS support with the partition migration.
When the migration initiates, the VAS migration handler will be
invoked before pseries_suspend() to close all active windows and
mark them in-active with VAS_WIN_MIGRATE_CLOSE status. Whereas
this migration handler is called after migration to reopen all
windows which has VAS_WIN_MIGRATE_CLOSE status and make them
active again. The user space gets paste instruction failure
when it sends requests on these in-active windows.
These patches depend on VAS/DLPAR support patch series
Haren Myneni (3):
powerpc/pseries/vas: Modify reconfig open/close functions for
migration
powerpc/pseries/vas: Add VAS migration handler
powerpc/pseries/vas: Disable window open during migration
arch/powerpc/include/asm/vas.h | 2 +
arch/powerpc/platforms/pseries/mobility.c | 5 +
arch/powerpc/platforms/pseries/vas.c | 187 +++++++++++++++++++---
arch/powerpc/platforms/pseries/vas.h | 6 +
4 files changed, 181 insertions(+), 19 deletions(-)
--
2.27.0
^ permalink raw reply
* [PATCH 1/3] powerpc/pseries/vas: Modify reconfig open/close functions for migration
From: Haren Myneni @ 2022-01-29 1:19 UTC (permalink / raw)
To: mpe, linuxppc-dev, npiggin, nathanl
In-Reply-To: <3ac406c37f4aa17e742325b5a86e073fd944428b.camel@linux.ibm.com>
VAS is a hardware engine stays on the chip. So when the partition
migrates, all VAS windows on the source system have to be closed
and reopen them on the destination after migration.
This patch make changes to the current reconfig_open/close_windows
functions to support migration:
- Set VAS_WIN_MIGRATE_CLOSE to the window status when closes and
reopen windows with the same status during resume.
- Continue to close all windows even if deallocate HCALL failed
(should not happen) since no way to stop migration with the
current LPM implementation.
- If the DLPAR CPU event happens while migration is in progress,
set VAS_WIN_NO_CRED_CLOSE to the window status. Close window
happens with the first event (migration or DLPAR) and Reopen
window happens only with the last event (migration or DLPAR).
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
arch/powerpc/include/asm/vas.h | 2 +
arch/powerpc/platforms/pseries/vas.c | 88 ++++++++++++++++++++++------
2 files changed, 73 insertions(+), 17 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index ddc05a8fc2e3..f21e76f47175 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -42,6 +42,8 @@
/* Linux status bits */
#define VAS_WIN_NO_CRED_CLOSE 0x00000004 /* Window is closed due to */
/* lost credit */
+#define VAS_WIN_MIGRATE_CLOSE 0x00000008 /* Window is closed due to */
+ /* migration */
/*
* Get/Set bit fields
*/
diff --git a/arch/powerpc/platforms/pseries/vas.c b/arch/powerpc/platforms/pseries/vas.c
index 3400f4fc6609..e4797fc73553 100644
--- a/arch/powerpc/platforms/pseries/vas.c
+++ b/arch/powerpc/platforms/pseries/vas.c
@@ -456,11 +456,12 @@ static int vas_deallocate_window(struct vas_window *vwin)
mutex_lock(&vas_pseries_mutex);
/*
* VAS window is already closed in the hypervisor when
- * lost the credit. So just remove the entry from
- * the list, remove task references and free vas_window
+ * lost the credit or with migration. So just remove the entry
+ * from the list, remove task references and free vas_window
* struct.
*/
- if (win->vas_win.status & VAS_WIN_NO_CRED_CLOSE) {
+ if (!(win->vas_win.status & VAS_WIN_NO_CRED_CLOSE) &&
+ !(win->vas_win.status & VAS_WIN_MIGRATE_CLOSE)) {
rc = deallocate_free_window(win);
if (rc) {
mutex_unlock(&vas_pseries_mutex);
@@ -577,12 +578,14 @@ static int __init get_vas_capabilities(u8 feat, enum vas_cop_feat_type type,
* by setting the remapping to new paste address if the window is
* active.
*/
-static int reconfig_open_windows(struct vas_caps *vcaps, int creds)
+static int reconfig_open_windows(struct vas_caps *vcaps, int creds,
+ bool migrate)
{
long domain[PLPAR_HCALL9_BUFSIZE] = {VAS_DEFAULT_DOMAIN_ID};
struct vas_cop_feat_caps *caps = &vcaps->caps;
struct pseries_vas_window *win = NULL, *tmp;
int rc, mv_ents = 0;
+ int flag;
/*
* Nothing to do if there are no closed windows.
@@ -601,8 +604,10 @@ static int reconfig_open_windows(struct vas_caps *vcaps, int creds)
* (dedicated). If 1 core is added, this LPAR can have 20 more
* credits. It means the kernel can reopen 20 windows. So move
* 20 entries in the VAS windows lost and reopen next 20 windows.
+ * For partition migration, reopen all windows that are closed
+ * during resume.
*/
- if (vcaps->close_wins > creds)
+ if ((vcaps->close_wins > creds) && !migrate)
mv_ents = vcaps->close_wins - creds;
list_for_each_entry_safe(win, tmp, &vcaps->list, win_list) {
@@ -612,12 +617,35 @@ static int reconfig_open_windows(struct vas_caps *vcaps, int creds)
mv_ents--;
}
+ /*
+ * Open windows if they are closed only with migration or
+ * DLPAR (lost credit) before.
+ */
+ if (migrate)
+ flag = VAS_WIN_MIGRATE_CLOSE;
+ else
+ flag = VAS_WIN_NO_CRED_CLOSE;
+
list_for_each_entry_safe_from(win, tmp, &vcaps->list, win_list) {
+ /*
+ * This window is closed with DLPAR and migration events.
+ * So reopen the window with the last event.
+ * The user space is not suspended with the current
+ * migration notifier. So the user space can issue DLPAR
+ * CPU hotplug while migration in progress. In this case
+ * this window will be opened with the last event.
+ */
+ if ((win->vas_win.status & VAS_WIN_NO_CRED_CLOSE) &&
+ (win->vas_win.status & VAS_WIN_MIGRATE_CLOSE)) {
+ win->vas_win.status &= ~flag;
+ continue;
+ }
+
/*
* Nothing to do on this window if it is not closed
- * with VAS_WIN_NO_CRED_CLOSE
+ * with this flag
*/
- if (!(win->vas_win.status & VAS_WIN_NO_CRED_CLOSE))
+ if (!(win->vas_win.status & flag))
continue;
rc = allocate_setup_window(win, (u64 *)&domain[0],
@@ -633,7 +661,7 @@ static int reconfig_open_windows(struct vas_caps *vcaps, int creds)
/*
* Set window status to active
*/
- win->vas_win.status &= ~VAS_WIN_NO_CRED_CLOSE;
+ win->vas_win.status &= ~flag;
mutex_unlock(&win->vas_win.task_ref.mmap_mutex);
win->win_type = caps->win_type;
if (!--vcaps->close_wins)
@@ -660,20 +688,32 @@ static int reconfig_open_windows(struct vas_caps *vcaps, int creds)
* the user space to fall back to SW compression and manage with the
* existing windows.
*/
-static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds)
+static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds,
+ bool migrate)
{
struct pseries_vas_window *win, *tmp;
struct vas_user_win_ref *task_ref;
struct vm_area_struct *vma;
- int rc = 0;
+ int rc = 0, flag;
+
+ if (migrate)
+ flag = VAS_WIN_MIGRATE_CLOSE;
+ else
+ flag = VAS_WIN_NO_CRED_CLOSE;
list_for_each_entry_safe(win, tmp, &vcap->list, win_list) {
/*
* This window is already closed due to lost credit
- * before. Go for next window.
+ * or for migration before. Go for next window.
+ * For migration, nothing to do since this window
+ * closed for DLPAR and will be reopened even on
+ * the destination system with other DLPAR operation.
*/
- if (win->vas_win.status & VAS_WIN_NO_CRED_CLOSE)
+ if ((win->vas_win.status & VAS_WIN_MIGRATE_CLOSE) ||
+ (win->vas_win.status & VAS_WIN_NO_CRED_CLOSE)) {
+ win->vas_win.status |= flag;
continue;
+ }
task_ref = &win->vas_win.task_ref;
mutex_lock(&task_ref->mmap_mutex);
@@ -682,7 +722,7 @@ static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds)
* Number of available credits are reduced, So select
* and close windows.
*/
- win->vas_win.status |= VAS_WIN_NO_CRED_CLOSE;
+ win->vas_win.status |= flag;
mmap_write_lock(task_ref->mm);
/*
@@ -705,12 +745,24 @@ static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds)
* later when the process issued with close(FD).
*/
rc = deallocate_free_window(win);
- if (rc)
+ /*
+ * This failure is from the hypervisor.
+ * No way to stop migration for these failures.
+ * So ignore error and continue closing other windows.
+ */
+ if (rc && !migrate)
return rc;
vcap->close_wins++;
- if (!--excess_creds)
+ /*
+ * For migration, do not depend on lpar_creds in case if
+ * mismatch with the hypervisor value (should not happen).
+ * So close all active windows in the list and will be
+ * reopened windows based on the new lpar_creds on the
+ * destination system during resume.
+ */
+ if (!migrate && !--excess_creds)
break;
}
@@ -765,7 +817,8 @@ int vas_reconfig_capabilties(u8 type)
* target, reopen windows if they are closed due to
* the previous DLPAR (core removal).
*/
- rc = reconfig_open_windows(vcaps, new_creds - lpar_creds);
+ rc = reconfig_open_windows(vcaps, new_creds - lpar_creds,
+ false);
} else {
/*
* # active windows is more than new LPAR available
@@ -773,7 +826,8 @@ int vas_reconfig_capabilties(u8 type)
*/
active_wins = vcaps->num_wins - vcaps->close_wins;
if (active_wins > new_creds)
- rc = reconfig_close_windows(vcaps, active_wins - new_creds);
+ rc = reconfig_close_windows(vcaps, active_wins - new_creds,
+ false);
}
out:
--
2.27.0
^ permalink raw reply related
* [PATCH 2/3] powerpc/pseries/vas: Add VAS migration handler
From: Haren Myneni @ 2022-01-29 1:20 UTC (permalink / raw)
To: mpe, linuxppc-dev, npiggin, nathanl
In-Reply-To: <3ac406c37f4aa17e742325b5a86e073fd944428b.camel@linux.ibm.com>
Since the VAS windows belong to the VAS hardware resource, the
hypervisor expects the partition to close them on source partition
and reopen them after the partition migrated on the destination
machine.
This handler is called before pseries_suspend() to close these
windows and again invoked after migration. All active windows
for both default and QoS types will be closed and mark them
in-active and reopened after migration with this handler.
During the migration, the user space receives paste instruction
failure if it issues copy/paste on these in-active windows.
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
arch/powerpc/platforms/pseries/mobility.c | 5 ++
arch/powerpc/platforms/pseries/vas.c | 86 +++++++++++++++++++++++
arch/powerpc/platforms/pseries/vas.h | 6 ++
3 files changed, 97 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/mobility.c b/arch/powerpc/platforms/pseries/mobility.c
index 85033f392c78..70004243e25e 100644
--- a/arch/powerpc/platforms/pseries/mobility.c
+++ b/arch/powerpc/platforms/pseries/mobility.c
@@ -26,6 +26,7 @@
#include <asm/machdep.h>
#include <asm/rtas.h>
#include "pseries.h"
+#include "vas.h" /* vas_migration_handler() */
#include "../../kernel/cacheinfo.h"
static struct kobject *mobility_kobj;
@@ -669,12 +670,16 @@ static int pseries_migrate_partition(u64 handle)
if (ret)
return ret;
+ vas_migration_handler(VAS_SUSPEND);
+
ret = pseries_suspend(handle);
if (ret == 0)
post_mobility_fixup();
else
pseries_cancel_migration(handle, ret);
+ vas_migration_handler(VAS_RESUME);
+
return ret;
}
diff --git a/arch/powerpc/platforms/pseries/vas.c b/arch/powerpc/platforms/pseries/vas.c
index e4797fc73553..b53e3fe02971 100644
--- a/arch/powerpc/platforms/pseries/vas.c
+++ b/arch/powerpc/platforms/pseries/vas.c
@@ -873,6 +873,92 @@ static struct notifier_block pseries_vas_nb = {
.notifier_call = pseries_vas_notifier,
};
+/*
+ * For LPM, all windows have to be closed on the source partition
+ * before migration and reopen them on the destination partition
+ * after migration. So closing windows during suspend and
+ * reopen them during resume.
+ */
+int vas_migration_handler(int action)
+{
+ struct hv_vas_cop_feat_caps *hv_caps;
+ struct vas_cop_feat_caps *caps;
+ int lpar_creds, new_creds = 0;
+ struct vas_caps *vcaps;
+ int i, rc = 0;
+
+ hv_caps = kmalloc(sizeof(*hv_caps), GFP_KERNEL);
+ if (!hv_caps)
+ return -ENOMEM;
+
+ mutex_lock(&vas_pseries_mutex);
+
+ for (i = 0; i < VAS_MAX_FEAT_TYPE; i++) {
+ vcaps = &vascaps[i];
+ caps = &vcaps->caps;
+ lpar_creds = atomic_read(&caps->target_creds);
+
+ rc = h_query_vas_capabilities(H_QUERY_VAS_CAPABILITIES,
+ vcaps->feat,
+ (u64)virt_to_phys(hv_caps));
+ if (!rc) {
+ new_creds = be16_to_cpu(hv_caps->target_lpar_creds);
+ /*
+ * Should not happen. But incase print messages, close
+ * all windows in the list during suspend and reopen
+ * windows based on new lpar_creds on the destination
+ * system.
+ */
+ if (lpar_creds != new_creds) {
+ pr_err("state(%d): lpar creds: %d HV lpar creds: %d\n",
+ action, lpar_creds, new_creds);
+ pr_err("Used creds: %d, Active creds: %d\n",
+ atomic_read(&caps->used_creds),
+ vcaps->num_wins - vcaps->close_wins);
+ }
+ } else {
+ pr_err("state(%d): Get VAS capabilities failed with %d\n",
+ action, rc);
+ /*
+ * We can not stop migration with the current lpm
+ * implementation. So continue closing all windows in
+ * the list (during suspend) and return without
+ * opening windows (during resume) if VAS capabilities
+ * HCALL failed.
+ */
+ if (action == VAS_RESUME)
+ goto out;
+ }
+
+ switch (action) {
+ case VAS_SUSPEND:
+ rc = reconfig_close_windows(vcaps, vcaps->num_wins,
+ true);
+ break;
+ case VAS_RESUME:
+ atomic_set(&caps->target_creds, new_creds);
+ rc = reconfig_open_windows(vcaps, new_creds, true);
+ break;
+ default:
+ /* should not happen */
+ pr_err("Invalid migration action %d\n", action);
+ rc = -EINVAL;
+ goto out;
+ }
+
+ /*
+ * Ignore errors during suspend and return for resume.
+ */
+ if (rc && (action == VAS_RESUME))
+ goto out;
+ }
+
+out:
+ mutex_unlock(&vas_pseries_mutex);
+ kfree(hv_caps);
+ return rc;
+}
+
static int __init pseries_vas_init(void)
{
struct hv_vas_cop_feat_caps *hv_cop_caps;
diff --git a/arch/powerpc/platforms/pseries/vas.h b/arch/powerpc/platforms/pseries/vas.h
index 4cf1d0ef66a5..36dd140af01b 100644
--- a/arch/powerpc/platforms/pseries/vas.h
+++ b/arch/powerpc/platforms/pseries/vas.h
@@ -27,6 +27,11 @@
#define VAS_GZIP_QOS_CAPABILITIES 0x56516F73477A6970
#define VAS_GZIP_DEFAULT_CAPABILITIES 0x56446566477A6970
+enum vas_migrate_action {
+ VAS_SUSPEND,
+ VAS_RESUME,
+};
+
/*
* Co-processor feature - GZIP QoS windows or GZIP default windows
*/
@@ -127,4 +132,5 @@ struct pseries_vas_window {
int sysfs_add_vas_caps(struct vas_cop_feat_caps *caps);
int vas_reconfig_capabilties(u8 type);
int __init sysfs_pseries_vas_init(struct vas_all_caps *vas_caps);
+int vas_migration_handler(int action);
#endif /* _VAS_H */
--
2.27.0
^ permalink raw reply related
* [PATCH 3/3] powerpc/pseries/vas: Disable window open during migration
From: Haren Myneni @ 2022-01-29 1:21 UTC (permalink / raw)
To: mpe, linuxppc-dev, npiggin, nathanl
In-Reply-To: <3ac406c37f4aa17e742325b5a86e073fd944428b.camel@linux.ibm.com>
The current partition migration implementation does not freeze the
user space and the user space can continue open VAS windows. So
when migration_in_progress flag is enabled, VAS open window
API returns -EBUSY.
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
arch/powerpc/platforms/pseries/vas.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/vas.c b/arch/powerpc/platforms/pseries/vas.c
index b53e3fe02971..63316c401abb 100644
--- a/arch/powerpc/platforms/pseries/vas.c
+++ b/arch/powerpc/platforms/pseries/vas.c
@@ -29,6 +29,7 @@ static bool copypaste_feat;
static struct vas_caps vascaps[VAS_MAX_FEAT_TYPE];
static DEFINE_MUTEX(vas_pseries_mutex);
+static bool migration_in_progress;
static long hcall_return_busy_check(long rc)
{
@@ -355,8 +356,11 @@ static struct vas_window *vas_allocate_window(int vas_id, u64 flags,
* same fault IRQ is not freed by the OS before.
*/
mutex_lock(&vas_pseries_mutex);
- rc = allocate_setup_window(txwin, (u64 *)&domain[0],
- cop_feat_caps->win_type);
+ if (migration_in_progress)
+ rc = -EBUSY;
+ else
+ rc = allocate_setup_window(txwin, (u64 *)&domain[0],
+ cop_feat_caps->win_type);
mutex_unlock(&vas_pseries_mutex);
if (rc)
goto out;
@@ -893,6 +897,11 @@ int vas_migration_handler(int action)
mutex_lock(&vas_pseries_mutex);
+ if (action == VAS_SUSPEND)
+ migration_in_progress = true;
+ else
+ migration_in_progress = false;
+
for (i = 0; i < VAS_MAX_FEAT_TYPE; i++) {
vcaps = &vascaps[i];
caps = &vcaps->caps;
--
2.27.0
^ permalink raw reply related
* [PATCH] powerpc/fadump: Use swap() instead of open coding it
From: Jiapeng Chong @ 2022-01-29 3:48 UTC (permalink / raw)
To: mpe; +Cc: Jiapeng Chong, Abaci Robot, linux-kernel, paulus, linuxppc-dev
Clean the following coccicheck warning:
./arch/powerpc/kernel/fadump.c:1291:34-35: WARNING opportunity for
swap().
Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
---
arch/powerpc/kernel/fadump.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c
index d0ad86b67e66..de08dd078081 100644
--- a/arch/powerpc/kernel/fadump.c
+++ b/arch/powerpc/kernel/fadump.c
@@ -1271,7 +1271,6 @@ static void fadump_release_reserved_area(u64 start, u64 end)
static void sort_and_merge_mem_ranges(struct fadump_mrange_info *mrange_info)
{
struct fadump_memory_range *mem_ranges;
- struct fadump_memory_range tmp_range;
u64 base, size;
int i, j, idx;
@@ -1286,11 +1285,8 @@ static void sort_and_merge_mem_ranges(struct fadump_mrange_info *mrange_info)
if (mem_ranges[idx].base > mem_ranges[j].base)
idx = j;
}
- if (idx != i) {
- tmp_range = mem_ranges[idx];
- mem_ranges[idx] = mem_ranges[i];
- mem_ranges[i] = tmp_range;
- }
+ if (idx != i)
+ swap(mem_ranges[idx], mem_ranges[i]);
}
/* Merge adjacent reserved ranges */
--
2.20.1.7.g153144c
^ permalink raw reply related
* [PATCH] powerpc: platforms: 52xx: Fix a resource leak in an error handling path
From: Christophe JAILLET @ 2022-01-29 7:16 UTC (permalink / raw)
To: Anatolij Gustschin, Michael Ellerman, Benjamin Herrenschmidt,
Paul Mackerras, Grant Likely, John Bonesio
Cc: Christophe JAILLET, linuxppc-dev, kernel-janitors, linux-kernel
The error handling path of mpc52xx_lpbfifo_probe() and a request_irq() is
not balanced by a corresponding free_irq().
Add the missing call, as already done in the remove function.
Fixes: 3c9059d79f5e ("powerpc/5200: add LocalPlus bus FIFO device driver")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
---
Another strange thing is that the remove function has:
/* Release the bestcomm transmit task */
free_irq(bcom_get_task_irq(lpbfifo.bcom_tx_task), &lpbfifo);
but I've not been able to find a corresponding request_irq().
Is it dead code? Is there something missing in the probe?
(...Is it working?...)
---
arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c b/arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c
index b91ebebd9ff2..e0049b7df212 100644
--- a/arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c
+++ b/arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c
@@ -530,6 +530,7 @@ static int mpc52xx_lpbfifo_probe(struct platform_device *op)
err_bcom_rx_irq:
bcom_gen_bd_rx_release(lpbfifo.bcom_rx_task);
err_bcom_rx:
+ free_irq(lpbfifo.irq, &lpbfifo);
err_irq:
iounmap(lpbfifo.regs);
lpbfifo.regs = NULL;
--
2.32.0
^ permalink raw reply related
* [PATCH v2 0/2] KVM: PPC: Book3S PR: Fixes for AIL and SCV
From: Nicholas Piggin @ 2022-01-29 7:25 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
The first patch in this series fixes a KVM PR host crash due to a
guest executing the scv instruction or with a pseries SMP host, the
host CPUs executing the scv instruction while a PR guest is running.
The second patch fixes unimplemented H_SET_MODE AIL modes by returning
failure from the hcall rather than succeeding but not implementing
the required behaviour. This works around missing host scv support for
scv-capable Linux guests by causing them to disable the facility.
Still looking at doing a proper capability for QEMU/KVM so we can get
consistency between HV, PR, and TCG. That shouldn't change these
patches though.
Thanks,
Nick
Nicholas Piggin (2):
KVM: PPC: Book3S PR: Disable SCV when AIL could be disabled
KVM: PPC: Book3S PR: Disallow AIL != 0
arch/powerpc/kernel/exceptions-64s.S | 4 ++++
arch/powerpc/kernel/setup_64.c | 28 ++++++++++++++++++++++++++++
arch/powerpc/kvm/Kconfig | 9 +++++++++
arch/powerpc/kvm/book3s_pr.c | 20 ++++++++++++++------
arch/powerpc/kvm/book3s_pr_papr.c | 20 ++++++++++++++++++++
5 files changed, 75 insertions(+), 6 deletions(-)
--
2.23.0
^ permalink raw reply
* [PATCH v2 1/2] KVM: PPC: Book3S PR: Disable SCV when AIL could be disabled
From: Nicholas Piggin @ 2022-01-29 7:25 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
In-Reply-To: <20220129072511.105523-1-npiggin@gmail.com>
PR KVM does not support running with AIL enabled, and SCV does is not
supported with AIL disabled. Fix this by ensuring the SCV facility is
disabled with FSCR while a CPU could be running with AIL=0.
The PowerNV host supports disabling AIL on a per-CPU basis, so SCV just
needs to be disabled when a vCPU is being run.
The pSeries machine can only switch AIL on a system-wide basis, so it
must disable SCV support at boot if the configuration can potentially
run a PR KVM guest.
Also ensure a the FSCR[SCV] bit can not be enabled when emulating
mtFSCR for the guest.
SCV is not emulated for the PR guest at the moment, this just fixes the
host crashes.
Alternatives considered and rejected:
- SCV support can not be disabled by PR KVM after boot, because it is
advertised to userspace with HWCAP.
- AIL can not be disabled on a per-CPU basis. At least when running on
pseries it is a per-LPAR setting.
- Support for real-mode SCV vectors will not be added because they are
at 0x17000 so making such a large fixed head space causes immediate
value limits to be exceeded, requiring a lot rework and more code.
- Disabling SCV for any PR KVM possible kernel will cause a slowdown
when not using PR KVM.
- A boot time option to disable SCV to use PR KVM is user-hostile.
- System call instruction emulation for SCV facility unavailable
instructions is too complex and old emulation code was subtly broken
and removed.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/kernel/exceptions-64s.S | 4 ++++
arch/powerpc/kernel/setup_64.c | 28 ++++++++++++++++++++++++++++
arch/powerpc/kvm/Kconfig | 9 +++++++++
arch/powerpc/kvm/book3s_pr.c | 20 ++++++++++++++------
4 files changed, 55 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 55caeee37c08..b66dd6f775a4 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -809,6 +809,10 @@ __start_interrupts:
* - MSR_EE|MSR_RI is clear (no reentrant exceptions)
* - Standard kernel environment is set up (stack, paca, etc)
*
+ * KVM:
+ * These interrupts do not elevate HV 0->1, so HV is not involved. PR KVM
+ * ensures that FSCR[SCV] is disabled whenever it has to force AIL off.
+ *
* Call convention:
*
* syscall register convention is in Documentation/powerpc/syscall64-abi.rst
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index be8577ac9397..7f7da641e551 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -197,6 +197,34 @@ static void __init configure_exceptions(void)
/* Under a PAPR hypervisor, we need hypercalls */
if (firmware_has_feature(FW_FEATURE_SET_MODE)) {
+ /*
+ * - PR KVM does not support AIL mode interrupts in the host
+ * while a PR guest is running.
+ *
+ * - SCV system call interrupt vectors are only implemented for
+ * AIL mode interrupts.
+ *
+ * - On pseries, AIL mode can only be enabled and disabled
+ * system-wide so when a PR VM is created on a pseries host,
+ * all CPUs of the host are set to AIL=0 mode.
+ *
+ * - Therefore host CPUs must not execute scv while a PR VM
+ * exists.
+ *
+ * - SCV support can not be disabled dynamically because the
+ * feature is advertised to host userspace. Disabling the
+ * facility and emulating it would be possible but is not
+ * implemented.
+ *
+ * - So SCV support is blanket diabled if PR KVM could possibly
+ * run. That is, PR support compiled in, booting on pseries
+ * with hash MMU.
+ */
+ if (IS_ENABLED(CONFIG_KVM_BOOK3S_PR_POSSIBLE) && !radix_enabled()) {
+ init_task.thread.fscr &= ~FSCR_SCV;
+ cur_cpu_spec->cpu_user_features2 &= ~PPC_FEATURE2_SCV;
+ }
+
/* Enable AIL if possible */
if (!pseries_enable_reloc_on_exc()) {
init_task.thread.fscr &= ~FSCR_SCV;
diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index 18e58085447c..ddd88179110a 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -112,12 +112,21 @@ config KVM_BOOK3S_64_PR
guest in user mode (problem state) and emulating all
privileged instructions and registers.
+ This is only available for hash MMU mode and only supports
+ guests that use hash MMU mode.
+
This is not as fast as using hypervisor mode, but works on
machines where hypervisor mode is not available or not usable,
and can emulate processors that are different from the host
processor, including emulating 32-bit processors on a 64-bit
host.
+ Selecting this option will cause the SCV facility to be
+ disabled when the kernel is booted on the pseries platform in
+ hash MMU mode (regardless of PR VMs running). When any PR VMs
+ are running, "AIL" mode is disabled which may slow interrupts
+ and system calls on the host.
+
config KVM_BOOK3S_HV_EXIT_TIMING
bool "Detailed timing for hypervisor real-mode code"
depends on KVM_BOOK3S_HV_POSSIBLE && DEBUG_FS
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index 34a801c3604a..4d1c84b94b77 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -140,9 +140,12 @@ static void kvmppc_core_vcpu_load_pr(struct kvm_vcpu *vcpu, int cpu)
#endif
/* Disable AIL if supported */
- if (cpu_has_feature(CPU_FTR_HVMODE) &&
- cpu_has_feature(CPU_FTR_ARCH_207S))
- mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~LPCR_AIL);
+ if (cpu_has_feature(CPU_FTR_HVMODE)) {
+ if (cpu_has_feature(CPU_FTR_ARCH_207S))
+ mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~LPCR_AIL);
+ if (cpu_has_feature(CPU_FTR_ARCH_300) && (current->thread.fscr & FSCR_SCV))
+ mtspr(SPRN_FSCR, mfspr(SPRN_FSCR) & ~FSCR_SCV);
+ }
vcpu->cpu = smp_processor_id();
#ifdef CONFIG_PPC_BOOK3S_32
@@ -175,9 +178,12 @@ static void kvmppc_core_vcpu_put_pr(struct kvm_vcpu *vcpu)
kvmppc_save_tm_pr(vcpu);
/* Enable AIL if supported */
- if (cpu_has_feature(CPU_FTR_HVMODE) &&
- cpu_has_feature(CPU_FTR_ARCH_207S))
- mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) | LPCR_AIL_3);
+ if (cpu_has_feature(CPU_FTR_HVMODE)) {
+ if (cpu_has_feature(CPU_FTR_ARCH_207S))
+ mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) | LPCR_AIL_3);
+ if (cpu_has_feature(CPU_FTR_ARCH_300) && (current->thread.fscr & FSCR_SCV))
+ mtspr(SPRN_FSCR, mfspr(SPRN_FSCR) | FSCR_SCV);
+ }
vcpu->cpu = -1;
}
@@ -1037,6 +1043,8 @@ static int kvmppc_handle_fac(struct kvm_vcpu *vcpu, ulong fac)
void kvmppc_set_fscr(struct kvm_vcpu *vcpu, u64 fscr)
{
+ if (fscr & FSCR_SCV)
+ fscr &= ~FSCR_SCV; /* SCV must not be enabled */
if ((vcpu->arch.fscr & FSCR_TAR) && !(fscr & FSCR_TAR)) {
/* TAR got dropped, drop it in shadow too */
kvmppc_giveup_fac(vcpu, FSCR_TAR_LG);
--
2.23.0
^ permalink raw reply related
* [PATCH v2 2/2] KVM: PPC: Book3S PR: Disallow AIL != 0
From: Nicholas Piggin @ 2022-01-29 7:25 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
In-Reply-To: <20220129072511.105523-1-npiggin@gmail.com>
KVM PR does not implement address translation modes on interrupt, so it
must not allow H_SET_MODE to succeed. The behaviour change caused by
this mode is architected and not advisory (interrupts *must* behave
differently).
QEMU does not deal with differences in AIL support in the host. The
solution to that is a spapr capability and corresponding KVM CAP, but
this patch does not break things more than before (the host behaviour
already differs, this change just disallows some modes that are not
implemented properly).
By happy coincidence, this allows PR Linux guests that are using the SCV
facility to boot and run, because Linux disables the use of SCV if AIL
can not be set to 3. This does not fix the underlying problem of missing
SCV support (an OS could implement real-mode SCV vectors and try to
enable the facility). The true fix for that is for KVM PR to emulate scv
interrupts from the facility unavailable interrupt.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/kvm/book3s_pr_papr.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/arch/powerpc/kvm/book3s_pr_papr.c b/arch/powerpc/kvm/book3s_pr_papr.c
index 1f10e7dfcdd0..dc4f51ac84bc 100644
--- a/arch/powerpc/kvm/book3s_pr_papr.c
+++ b/arch/powerpc/kvm/book3s_pr_papr.c
@@ -281,6 +281,22 @@ static int kvmppc_h_pr_logical_ci_store(struct kvm_vcpu *vcpu)
return EMULATE_DONE;
}
+static int kvmppc_h_pr_set_mode(struct kvm_vcpu *vcpu)
+{
+ unsigned long mflags = kvmppc_get_gpr(vcpu, 4);
+ unsigned long resource = kvmppc_get_gpr(vcpu, 5);
+
+ if (resource == H_SET_MODE_RESOURCE_ADDR_TRANS_MODE) {
+ /* KVM PR does not provide AIL!=0 to guests */
+ if (mflags == 0)
+ kvmppc_set_gpr(vcpu, 3, H_SUCCESS);
+ else
+ kvmppc_set_gpr(vcpu, 3, H_UNSUPPORTED_FLAG_START - 63);
+ return EMULATE_DONE;
+ }
+ return EMULATE_FAIL;
+}
+
#ifdef CONFIG_SPAPR_TCE_IOMMU
static int kvmppc_h_pr_put_tce(struct kvm_vcpu *vcpu)
{
@@ -384,6 +400,8 @@ int kvmppc_h_pr(struct kvm_vcpu *vcpu, unsigned long cmd)
return kvmppc_h_pr_logical_ci_load(vcpu);
case H_LOGICAL_CI_STORE:
return kvmppc_h_pr_logical_ci_store(vcpu);
+ case H_SET_MODE:
+ return kvmppc_h_pr_set_mode(vcpu);
case H_XIRR:
case H_CPPR:
case H_EOI:
@@ -421,6 +439,7 @@ int kvmppc_hcall_impl_pr(unsigned long cmd)
case H_CEDE:
case H_LOGICAL_CI_LOAD:
case H_LOGICAL_CI_STORE:
+ case H_SET_MODE:
#ifdef CONFIG_KVM_XICS
case H_XIRR:
case H_CPPR:
@@ -447,6 +466,7 @@ static unsigned int default_hcall_list[] = {
H_BULK_REMOVE,
H_PUT_TCE,
H_CEDE,
+ H_SET_MODE,
#ifdef CONFIG_KVM_XICS
H_XIRR,
H_CPPR,
--
2.23.0
^ permalink raw reply related
* Re: [PATCH] powerpc: platforms: 52xx: Fix a resource leak in an error handling path
From: Anatolij Gustschin @ 2022-01-29 9:50 UTC (permalink / raw)
To: Christophe JAILLET
Cc: kernel-janitors, linux-kernel, Paul Mackerras, John Bonesio,
linuxppc-dev
In-Reply-To: <dec1496d46ccd5311d0f6e9f9ca4238be11bf6a6.1643440531.git.christophe.jaillet@wanadoo.fr>
On Sat, 29 Jan 2022 08:16:04 +0100
Christophe JAILLET christophe.jaillet@wanadoo.fr wrote:
>The error handling path of mpc52xx_lpbfifo_probe() and a request_irq() is
>not balanced by a corresponding free_irq().
>
>Add the missing call, as already done in the remove function.
>
>Fixes: 3c9059d79f5e ("powerpc/5200: add LocalPlus bus FIFO device driver")
>Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
>---
>Another strange thing is that the remove function has:
> /* Release the bestcomm transmit task */
> free_irq(bcom_get_task_irq(lpbfifo.bcom_tx_task), &lpbfifo);
>but I've not been able to find a corresponding request_irq().
This driver does not request the tx task irq itself, but a fifo
client driver can request/free tx interrupts for submitted
fifo write tasks, like mpc5200 fec and pata drivers do, so
it is okay.
>Is it dead code? Is there something missing in the probe?
No.
--
Anatolij
^ permalink raw reply
* Re: [PATCH V2] powerpc/perf: Fix power_pmu_disable to call clear_pmi_irq_pending only if PMI is pending
From: Michael Ellerman @ 2022-01-29 11:14 UTC (permalink / raw)
To: mpe, Athira Rajeev; +Cc: kjain, maddy, linuxppc-dev, npiggin, rnsastry
In-Reply-To: <20220122033429.25395-1-atrajeev@linux.vnet.ibm.com>
On Sat, 22 Jan 2022 09:04:29 +0530, Athira Rajeev wrote:
> Running selftest with CONFIG_PPC_IRQ_SOFT_MASK_DEBUG enabled in kernel
> triggered below warning:
>
> [ 172.851380] ------------[ cut here ]------------
> [ 172.851391] WARNING: CPU: 8 PID: 2901 at arch/powerpc/include/asm/hw_irq.h:246 power_pmu_disable+0x270/0x280
> [ 172.851402] Modules linked in: dm_mod bonding nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables rfkill nfnetlink sunrpc xfs libcrc32c pseries_rng xts vmx_crypto uio_pdrv_genirq uio sch_fq_codel ip_tables ext4 mbcache jbd2 sd_mod t10_pi sg ibmvscsi ibmveth scsi_transport_srp fuse
> [ 172.851442] CPU: 8 PID: 2901 Comm: lost_exception_ Not tainted 5.16.0-rc5-03218-g798527287598 #2
> [ 172.851451] NIP: c00000000013d600 LR: c00000000013d5a4 CTR: c00000000013b180
> [ 172.851458] REGS: c000000017687860 TRAP: 0700 Not tainted (5.16.0-rc5-03218-g798527287598)
> [ 172.851465] MSR: 8000000000029033 <SF,EE,ME,IR,DR,RI,LE> CR: 48004884 XER: 20040000
> [ 172.851482] CFAR: c00000000013d5b4 IRQMASK: 1
> [ 172.851482] GPR00: c00000000013d5a4 c000000017687b00 c000000002a10600 0000000000000004
> [ 172.851482] GPR04: 0000000082004000 c0000008ba08f0a8 0000000000000000 00000008b7ed0000
> [ 172.851482] GPR08: 00000000446194f6 0000000000008000 c00000000013b118 c000000000d58e68
> [ 172.851482] GPR12: c00000000013d390 c00000001ec54a80 0000000000000000 0000000000000000
> [ 172.851482] GPR16: 0000000000000000 0000000000000000 c000000015d5c708 c0000000025396d0
> [ 172.851482] GPR20: 0000000000000000 0000000000000000 c00000000a3bbf40 0000000000000003
> [ 172.851482] GPR24: 0000000000000000 c0000008ba097400 c0000000161e0d00 c00000000a3bb600
> [ 172.851482] GPR28: c000000015d5c700 0000000000000001 0000000082384090 c0000008ba0020d8
> [ 172.851549] NIP [c00000000013d600] power_pmu_disable+0x270/0x280
> [ 172.851557] LR [c00000000013d5a4] power_pmu_disable+0x214/0x280
> [ 172.851565] Call Trace:
> [ 172.851568] [c000000017687b00] [c00000000013d5a4] power_pmu_disable+0x214/0x280 (unreliable)
> [ 172.851579] [c000000017687b40] [c0000000003403ac] perf_pmu_disable+0x4c/0x60
> [ 172.851588] [c000000017687b60] [c0000000003445e4] __perf_event_task_sched_out+0x1d4/0x660
> [ 172.851596] [c000000017687c50] [c000000000d1175c] __schedule+0xbcc/0x12a0
> [ 172.851602] [c000000017687d60] [c000000000d11ea8] schedule+0x78/0x140
> [ 172.851608] [c000000017687d90] [c0000000001a8080] sys_sched_yield+0x20/0x40
> [ 172.851615] [c000000017687db0] [c0000000000334dc] system_call_exception+0x18c/0x380
> [ 172.851622] [c000000017687e10] [c00000000000c74c] system_call_common+0xec/0x268
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/perf: Fix power_pmu_disable to call clear_pmi_irq_pending only if PMI is pending
https://git.kernel.org/powerpc/c/fb6433b48a178d4672cb26632454ee0b21056eaa
cheers
^ permalink raw reply
* Re: [PATCH] powerpc/fixmap: Fix VM debug warning on unmap
From: Michael Ellerman @ 2022-01-29 11:14 UTC (permalink / raw)
To: Christophe Leroy, Paul Mackerras, Michael Ellerman,
Benjamin Herrenschmidt
Cc: Maxime Bizon, linuxppc-dev, linux-kernel
In-Reply-To: <b0b752f6f6ecc60653e873f385c6f0dce4e9ab6a.1638789098.git.christophe.leroy@csgroup.eu>
On Mon, 6 Dec 2021 11:11:51 +0000, Christophe Leroy wrote:
> Unmapping a fixmap entry is done by calling __set_fixmap()
> with FIXMAP_PAGE_CLEAR as flags.
>
> Today, powerpc __set_fixmap() calls map_kernel_page().
>
> map_kernel_page() is not happy when called a second time
> for the same page.
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/fixmap: Fix VM debug warning on unmap
https://git.kernel.org/powerpc/c/aec982603aa8cc0a21143681feb5f60ecc69d718
cheers
^ permalink raw reply
* Re: (subset) [PATCH 1/3] powerpc/64s/interrupt: Fix decrementer storm
From: Michael Ellerman @ 2022-01-29 11:14 UTC (permalink / raw)
To: Nicholas Piggin, linuxppc-dev; +Cc: Alexey Kardashevskiy
In-Reply-To: <20220124143930.3923442-1-npiggin@gmail.com>
On Tue, 25 Jan 2022 00:39:28 +1000, Nicholas Piggin wrote:
> The decrementer exception can fail to be cleared when the interrupt
> returns in the case where the decrementer wraps with the next timer
> still beyond decrementer_max. This results in a decrementer interrupt
> storm. This is triggerable with small decrementer system with hard
> and soft watchdogs disabled.
>
> Fix this by always programming the decrementer if there was no timer.
>
> [...]
Patch 1 applied to powerpc/fixes.
[1/3] powerpc/64s/interrupt: Fix decrementer storm
https://git.kernel.org/powerpc/c/8defc2a5dd8f4c0cb19ecbaca8d3e89ab98524da
cheers
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox