From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Jones Subject: [PATCH kvm-unit-tests v2 2/2] arm/arm64: smp: add deadlock detection Date: Fri, 2 Jun 2017 17:31:09 +0200 Message-ID: <20170602153109.2904-3-drjones@redhat.com> References: <20170602153109.2904-1-drjones@redhat.com> Cc: pbonzini@redhat.com, rkrcmar@redhat.com To: kvm@vger.kernel.org Return-path: Received: from mx1.redhat.com ([209.132.183.28]:38408 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751154AbdFBPbW (ORCPT ); Fri, 2 Jun 2017 11:31:22 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C9748048F for ; Fri, 2 Jun 2017 15:31:21 +0000 (UTC) In-Reply-To: <20170602153109.2904-1-drjones@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: on_cpu() and friends are a bit risky when implemented without IPIs (no preemption), because we can easily get deadlocks. Luckily, it's also easy to detect those deadlocks, and assert, to at least make them easier to debug. Signed-off-by: Andrew Jones --- lib/arm/smp.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/lib/arm/smp.c b/lib/arm/smp.c index b4b43237e32e..bb999243de63 100644 --- a/lib/arm/smp.c +++ b/lib/arm/smp.c @@ -76,9 +76,24 @@ typedef void (*on_cpu_func)(void *); struct on_cpu_info { on_cpu_func func; void *data; + cpumask_t waiters; }; static struct on_cpu_info on_cpu_info[NR_CPUS]; +static void cpu_wait(int cpu) +{ + int me = smp_processor_id(); + + if (cpu == me) + return; + + cpumask_set_cpu(me, &on_cpu_info[cpu].waiters); + assert_msg(!cpumask_test_cpu(cpu, &on_cpu_info[me].waiters), "CPU%d <=> CPU%d deadlock detected", me, cpu); + while (!cpu_idle(cpu)) + wfe(); + cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters); +} + void do_idle(void) { int cpu = smp_processor_id(); @@ -117,8 +132,7 @@ void on_cpu_async(int cpu, void (*func)(void *data), void *data) spin_unlock(&lock); for (;;) { - while (!cpu_idle(cpu)) - wfe(); + cpu_wait(cpu); spin_lock(&lock); if ((volatile void *)on_cpu_info[cpu].func == NULL) break; @@ -134,9 +148,7 @@ void on_cpu_async(int cpu, void (*func)(void *data), void *data) void on_cpu(int cpu, void (*func)(void *data), void *data) { on_cpu_async(cpu, func, data); - - while (!cpu_idle(cpu)) - wfe(); + cpu_wait(cpu); } void on_cpus(void (*func)(void)) @@ -150,6 +162,14 @@ void on_cpus(void (*func)(void)) } func(); + for_each_present_cpu(cpu) { + if (cpu == me) + continue; + cpumask_set_cpu(me, &on_cpu_info[cpu].waiters); + assert_msg(!cpumask_test_cpu(cpu, &on_cpu_info[me].waiters), "CPU%d <=> CPU%d deadlock detected", me, cpu); + } while (cpumask_weight(&cpu_idle_mask) < nr_cpus - 1) wfe(); + for_each_present_cpu(cpu) + cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters); } -- 2.9.4